Image coding method and apparatus using motion vector

ABSTRACT

According to embodiments according to the present document, merge mode motion vector differences (MMVD) and symmetric motion vector differences (SMVD) may be performed on the basis of reference picture types and, in particular, short-term reference pictures may be used. Accordingly, prediction performance and coding efficiency in an inter prediction mode may be increased.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

The present document is related to an image coding method and apparatususing motion vector.

Related Art

Recently, demand for high-resolution, high-quality image/video such as4K or 8K or higher ultra high definition (UHD) image/video has increasedin various fields. As image/video data has high resolution and highquality, the amount of information or bits to be transmitted increasesrelative to the existing image/video data, and thus, transmitting imagedata using a medium such as an existing wired/wireless broadband line oran existing storage medium or storing image/video data using existingstorage medium increase transmission cost and storage cost.

In addition, interest and demand for immersive media such as virtualreality (VR) and artificial reality (AR) content or holograms hasrecently increased and broadcasting for image/video is havingcharacteristics different from reality images such as game images hasincreased.

Accordingly, a highly efficient image/video compression technology isrequired to effectively compress, transmit, store, and reproduceinformation of a high-resolution, high-quality image/video havingvarious characteristics as described above.

Furthermore, inter prediction in image/video coding may include aprocedure for symmetric motion vector difference (SMVD) referenceindices and/or a procedure for a merge motion vector difference (MMVD),and a technology for performing the procedures by considering referencepicture marking (e.g., refer to a short term or long term) is discussed.

SUMMARY

According to an embodiment of the present document, a method and adevice for improving image/video coding efficiency are provided.

According to an embodiment of the present document, a method and adevice for performing inter prediction efficiently in an image/videocoding system are provided.

According to an embodiment of the present document, a method and adevice for signaling information on motion vector differences for interprediction are provided.

According to an embodiment of the present document, a method and adevice for signaling information on L0 motion vector differences and L1motion vector differences are provided when bi-prediction is applied toa current block.

According to an embodiment of the present document, a method and adevice for signaling an SMVD flag are provided.

According to an embodiment of this document, prediction procedure may beperformed based on the type of reference picture for bi-prediction.

According to an embodiment of this document, a procedure about SMVDreference indices may be performed based on the reference picturemarking.

According to an embodiment of this document, a procedure about SMVDreference indices may be performed using short-term reference pictures(pictures marked as being used for a short-term reference).

According to an embodiment of the present document, a video/imagedecoding method performed by a decoding apparatus is provided.

According to an embodiment of the present document, a decoding apparatusfor performing video/image decoding is provided.

According to an embodiment of the present document, a video/imageencoding method performed by an encoding apparatus is provided.

According to an embodiment of the present document, an encodingapparatus for performing video/image encoding is provided.

According to one embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded video/imageinformation, generated according to the video/image encoding methoddisclosed in at least one of the embodiments of the present document, isstored.

According to an embodiment of the present document, there is provided acomputer-readable digital storage medium in which encoded information orencoded video/image information, causing to perform the video/imagedecoding method disclosed in at least one of the embodiments of thepresent document by the decoding apparatus, is stored.

Effects of the Disclosure

According to the present disclosure, the overall image/video compressioncan be efficiency improved.

According to the present disclosure, signaling of information on motionvector differences can be efficiency improved.

According to the present disclosure, L1 motion vector differences may beefficiently derived when bi-prediction is applied to a current block.

According to the present disclosure, information used for deriving L1motion vector differences may be signaled used on the types of referencepictures, and thus the complexity of a coding system may be reduced.

According to one embodiment of the present disclosure, efficient interprediction may be performed using short term reference pictures forderiving a reference picture index of SMVD.

The technical effects achieved through specific embodiments of thepresent disclosure are not limited to those described above. Forexample, various other technical effects may be obtained, which may beunderstood or derived from the present disclosure by a person havingordinary skills in the related art. Therefore, specific effects of thepresent disclosure are not limited to the embodiments disclosedexplicitly in this document and may include various other effects thatmay be understood or derived from the technical characteristics of thepresent disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates an example of a video/image codingsystem which may be applied to embodiments of this document.

FIG. 2 is a diagram schematically describing a configuration of avideo/image encoding apparatus which may be applied to embodiments ofthis document.

FIG. 3 is a diagram schematically describing a configuration of avideo/image decoding apparatus which may be applied to embodiments ofthis document.

FIG. 4 illustrates an example of an inter prediction-based video/imageencoding method.

FIG. 5 illustrates an example of an inter prediction-based video/imagedecoding method.

FIG. 6 exemplarily illustrates an inter prediction procedure.

FIG. 7 schematically illustrates a method of configuring a mergecandidate list according to this document.

FIG. 8 schematically illustrates a method of configuring an MVPcandidate list according to this document.

FIG. 9 is a diagram for describing an SMVD.

FIG. 10 is a diagram for describing a method of deriving a motion vectorin inter prediction.

FIG. 11 illustrates a process of deriving an MVD of an MMVD according toan embodiment of this document.

FIG. 12 illustrates a process of deriving an MVD of an MMVD according toanother embodiment of this document.

FIG. 13 illustrates a process of deriving an MVD of an MMVD according toanother embodiment of this document.

FIG. 14 illustrates a process of deriving an MVD of an MMVD according toan embodiment of this document.

FIG. 15 illustrates a process of deriving an MVD of an MMVD according toan embodiment of this document.

FIG. 16 is a diagram for describing an SMVD according to an embodimentof this document.

FIG. 17 is a flowchart illustrating an MMVD derivation method accordingto an embodiment of this document.

FIG. 18 is a flowchart illustrating an MMVD derivation method accordingto an embodiment of this document.

FIG. 19 is a flowchart illustrating an MMVD derivation method accordingto an embodiment of this document.

FIGS. 20 and 21 schematically illustrate an example of a video/imageencoding method and related components according to an embodiment(s) ofthis document.

FIGS. 22 and 23 schematically illustrate an example of an image/videodecoding method and related components according to an embodiment(s) ofthis document.

FIG. 24 illustrates an example of a content streaming system to whichembodiments disclosed in this document may be applied.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

The present document may be modified in various forms, and specificembodiments thereof are described and shown in the drawings. However,the embodiments are not intended for limiting the present document. Theterms used in the following description are used to merely describespecific embodiments, but are not intended to limit the presentdocument. An expression of a singular number includes an expression ofthe plural number, so long as it is clearly read differently. The termssuch as “include” and “have” are intended to indicate that features,numbers, steps, operations, elements, components, or combinationsthereof used in the following description exist and it should be thusunderstood that the possibility of existence or addition of one or moredifferent features, numbers, steps, operations, elements, components, orcombinations thereof is not excluded.

Meanwhile, each configuration in the drawings described in the presentdocument is shown independently for the convenience of descriptionregarding different characteristic functions, and does not mean thateach configuration is implemented as separate hardware or separatesoftware. For example, two or more components among each component maybe combined to form one component, or one component may be divided intoa plurality of components. Embodiments in which each component isintegrated and/or separated are also included in the scope of thedisclosure of the present document.

Hereinafter, examples of the present embodiment are described in detailwith reference to the accompanying drawings. In addition, like referencenumerals are used to indicate like elements throughout the drawings, andthe same descriptions on the like elements are omitted.

FIG. 1 illustrates an example of a video/image coding system to whichthe embodiments of the present document may be applied.

Referring to FIG. 1 , a video/image coding system may include a firstdevice (a source device) and a second device (a reception device). Thesource device may transmit encoded video/image information or data tothe reception device through a digital storage medium or network in theform of a file or streaming.

The source device may include a video source, an encoding apparatus, anda transmitter. The receiving device may include a receiver, a decodingapparatus, and a renderer. The encoding apparatus may be called avideo/image encoding apparatus, and the decoding apparatus may be calleda video/image decoding apparatus. The transmitter may be included in theencoding apparatus. The receiver may be included in the decodingapparatus. The renderer may include a display, and the display may beconfigured as a separate device or an external component.

The video source may acquire video/image through a process of capturing,synthesizing, or generating the video/image. The video source mayinclude a video/image capture device and/or a video/image generatingdevice. The video/image capture device may include, for example, one ormore cameras, video/image archives including previously capturedvideo/images, and the like. The video/image generating device mayinclude, for example, computers, tablets and smartphones, and may(electronically) generate video/images. For example, a virtualvideo/image may be generated through a computer or the like. In thiscase, the video/image capturing process may be replaced by a process ofgenerating related data.

The encoding apparatus may encode input video/image. The encodingapparatus may perform a series of procedures such as prediction,transform, and quantization for compaction and coding efficiency. Theencoded data (encoded video/image information) may be output in the formof a bitstream.

The transmitter may transmit the encoded image/image information or dataoutput in the form of a bitstream to the receiver of the receivingdevice through a digital storage medium or a network in the form of afile or streaming. The digital storage medium may include variousstorage mediums such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, and thelike. The transmitter may include an element for generating a media filethrough a predetermined file format and may include an element fortransmission through a broadcast/communication network. The receiver mayreceive/extract the bitstream and transmit the received bitstream to thedecoding apparatus.

The decoding apparatus may decode the video/image by performing a seriesof procedures such as dequantization, inverse transform, and predictioncorresponding to the operation of the encoding apparatus.

The renderer may render the decoded video/image. The renderedvideo/image may be displayed through the display.

The present document relates to video/image coding. For example, amethod/embodiment disclosed in the present document may be applied to amethod disclosed in the versatile video coding (VVC) standard, theessential video coding (EVC) standard, the AOMedia Video 1 (AV1)standard, the 2nd generation of audio video coding standard (AVS2) orthe next generation video/image coding standard (e.g., H.267, H.268, orthe like).

The present document suggests various embodiments of video/image coding,and the above embodiments may also be performed in combination with eachother unless otherwise specified.

In the present document, a video may refer to a series of images overtime. A picture generally refers to the unit representing one image at aparticular time frame, and a slice/tile refers to the unit constitutinga part of the picture in terms of coding. A slice/tile may include oneor more coding tree units (CTUs). One picture may consist of one or moreslices/tiles. One picture may consist of one or more tile groups. Onetile group may include one or more tiles. A brick may represent arectangular region of CTU rows within a tile in a picture. A tile may bepartitioned into a multiple bricks, each of which may be constructedwith one or more CTU rows within the tile. A tile that is notpartitioned into multiple bricks may also be referred to as a brick. Abrick scan may represent a specific sequential ordering of CTUspartitioning a picture, wherein the CTUs may be ordered in a CTU rasterscan within a brick, and bricks within a tile may be orderedconsecutively in a raster scan of the bricks of the tile, and tiles in apicture may be ordered consecutively in a raster scan of the tiles ofthe picture. A tile is a rectangular region of CTUs within a particulartile column and a particular tile row in a picture. The tile column is arectangular region of CTUs having a height equal to the height of thepicture and a width specified by syntax elements in the pictureparameter set. The tile row is a rectangular region of CTUs having aheight specified by syntax elements in the picture parameter set and awidth equal to the width of the picture. A tile scan is a specificsequential ordering of CTUs partitioning a picture in which the CTUs areordered consecutively in CTU raster scan in a tile whereas tiles in apicture are ordered consecutively in a raster scan of the tiles of thepicture. A slice includes an integer number of bricks of a picture thatmay be exclusively contained in a single NAL unit. A slice may consistof either a number of complete tiles or only a consecutive sequence ofcomplete bricks of one tile. In the present document, a tile group and aslice may be used in place of each other. For example, in the presentdocument, a tile group/tile group header may be referred to as aslice/slice header.

Meanwhile, one picture may be divided into two or more subpictures. Asubpicture may be a rectangular region of one or more slices within apicture.

A pixel or a pel may mean a smallest unit constituting one picture (orimage). Also, ‘sample’ may be used as a term corresponding to a pixel. Asample may generally represent a pixel or a value of a pixel, and mayrepresent only a pixel/pixel value of a luma component or only apixel/pixel value of a chroma component.

A unit may represent a basic unit of image processing. The unit mayinclude at least one of a specific region of the picture and informationrelated to the region. One unit may include one luma block and twochroma (ex. cb, cr) blocks. The unit may be used interchangeably withterms such as block or area in some cases. In a general case, an M×Nblock may include samples (or sample arrays) or a set (or array) oftransform coefficients of M columns and N rows. Alternatively, thesample may mean a pixel value in the spatial domain, and when such apixel value is transformed to the frequency domain, it may mean atransform coefficient in the frequency domain.

In the present document, “A or B” may mean “only A”, “only B” or “both Aand B”. In other words, “A or B” in the present document may beinterpreted as “A and/or B”. For example, in the present document “A, Bor C (A, B or C)” means “only A”, “only B”, “only C”, or “anycombination of A, B and C”.

A slash (/) or comma (comma) used in the present document may mean“and/or”. For example, “A/B” may mean “A and/or B”. Accordingly, “A/B”may mean “only A”, “only B”, or “both A and B”. For example, “A, B, C”may mean “A, B, or C”.

In the present document, “at least one of A and B” may mean “only A”,“only B” or “both A and B”. Also, in the present document, theexpression “at least one of A or B” or “at least one of A and/or B” maybe interpreted the same as “at least one of A and B”.

Also, in the present document, “at least one of A, B and C” means “onlyA”, “only B”, “only C”, or “any combination of A, B and C”. Also, “atleast one of A, B or C” or “at least one of A, B and/or C” may mean “atleast one of A, B and C”.

Also, parentheses used in the present document may mean “for example”.Specifically, when “prediction (intra prediction)” is indicated, “intraprediction” may be proposed as an example of “prediction”. In otherwords, “prediction” in the present document is not limited to “intraprediction”, and “intra prediction” may be proposed as an example of“prediction”. Also, even when “prediction (i.e., intra prediction)” isindicated, “intra prediction” may be proposed as an example of“prediction”.

Technical features that are individually described in one drawing in thepresent document may be implemented individually or simultaneously.

FIG. 2 is a diagram schematically illustrating a configuration of avideo/image encoding apparatus to which the embodiments of the presentdocument may be applied. Hereinafter, what is referred to as the videoencoding apparatus may include an image encoding apparatus.

Referring to FIG. 2 , the encoding apparatus 200 includes an imagepartitioner 210, a predictor 220, a residual processor 230, and anentropy encoder 240, an adder 250, a filter 260, and a memory 270. Thepredictor 220 may include an inter predictor 221 and an intra predictor222. The residual processor 230 may include a transformer 232, aquantizer 233, a dequantizer 234, and an inverse transformer 235. Theresidual processor 230 may further include a subtractor 231. The adder250 may be called a reconstructor or a reconstructed block generator.The image partitioner 210, the predictor 220, the residual processor230, the entropy encoder 240, the adder 250, and the filter 260 may beconfigured by at least one hardware component (ex. An encoder chipset orprocessor) according to an embodiment. In addition, the memory 270 mayinclude a decoded picture buffer (DPB) or may be configured by a digitalstorage medium. The hardware component may further include the memory270 as an internal/external component.

The image partitioner 210 may partition an input image (or a picture ora frame) input to the encoding apparatus 200 into one or moreprocessors. For example, the processor may be called a coding unit (CU).In this case, the coding unit may be recursively partitioned accordingto a quad-tree binary-tree ternary-tree (QTBTTT) structure from a codingtree unit (CTU) or a largest coding unit (LCU). For example, one codingunit may be partitioned into a plurality of coding units of a deeperdepth based on a quad tree structure, a binary tree structure, and/or aternary structure. In this case, for example, the quad tree structuremay be applied first and the binary tree structure and/or ternarystructure may be applied later. Alternatively, the binary tree structuremay be applied first. The coding procedure according to the presentdisclosure may be performed based on the final coding unit that is nolonger partitioned. In this case, the largest coding unit may be used asthe final coding unit based on coding efficiency according to imagecharacteristics, or if necessary, the coding unit may be recursivelypartitioned into coding units of deeper depth and a coding unit havingan optimal size may be used as the final coding unit. Here, the codingprocedure may include a procedure of prediction, transform, andreconstruction, which are described later. As another example, theprocessor may further include a prediction unit (PU) or a transform unit(TU). In this case, the prediction unit and the transform unit may besplit or partitioned from the aforementioned final coding unit. Theprediction unit may be a unit of sample prediction, and the transformunit may be a unit for deriving a transform coefficient and/or a unitfor deriving a residual signal from the transform coefficient.

The unit may be used interchangeably with terms such as block or area insome cases. In a general case, an M×N block may represent a set ofsamples or transform coefficients composed of M columns and N rows. Asample may generally represent a pixel or a value of a pixel, mayrepresent only a pixel/pixel value of a luma component or represent onlya pixel/pixel value of a chroma component. A sample may be used as aterm corresponding to one picture (or image) for a pixel or a pel.

In the encoding apparatus 200, a prediction signal (predicted block,prediction sample array) output from the inter predictor 221 or theintra predictor 222 is subtracted from an input image signal (originalblock, original sample array) to generate a residual signal residualblock, residual sample array), and the generated residual signal istransmitted to the transformer 232. In this case, as shown, a unit forsubtracting a prediction signal (predicted block, prediction samplearray) from the input image signal (original block, original samplearray) in the encoder 200 may be called a subtractor 231. The predictormay perform prediction on a block to be processed (hereinafter, referredto as a current block) and generate a predicted block includingprediction samples for the current block. The predictor may determinewhether intra prediction or inter prediction is applied on a currentblock or CU basis. As described later in the description of eachprediction mode, the predictor may generate various information relatedto prediction, such as prediction mode information, and transmit thegenerated information to the entropy encoder 240. The information on theprediction may be encoded in the entropy encoder 240 and output in theform of a bitstream.

The intra predictor 222 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The non-directional mode may include, for example,a DC mode and a planar mode. The directional mode may include, forexample, 33 directional prediction modes or 65 directional predictionmodes according to the degree of detail of the prediction direction.However, this is merely an example, more or less directional predictionmodes may be used depending on a setting. The intra predictor 222 maydetermine the prediction mode applied to the current block by using aprediction mode applied to a neighboring block.

The inter predictor 221 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. Here, in order to reduce theamount of motion information transmitted in the inter prediction mode,the motion information may be predicted in units of blocks, sub-blocks,or samples based on correlation of motion information between theneighboring block and the current block. The motion information mayinclude a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0prediction, L1 prediction, Bi prediction, etc.) information. In the caseof inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. The referencepicture including the reference block and the reference pictureincluding the temporal neighboring block may be the same or different.The temporal neighboring block may be called a collocated referenceblock, a co-located CU (colCU), and the like, and the reference pictureincluding the temporal neighboring block may be called a collocatedpicture (colPic). For example, the inter predictor 221 may configure amotion information candidate list based on neighboring blocks andgenerate information indicating which candidate is used to derive amotion vector and/or a reference picture index of the current block.Inter prediction may be performed based on various prediction modes. Forexample, in the case of a skip mode and a merge mode, the interpredictor 221 may use motion information of the neighboring block asmotion information of the current block. In the skip mode, unlike themerge mode, the residual signal may not be transmitted. In the case ofthe motion vector prediction (MVP) mode, the motion vector of theneighboring block may be used as a motion vector predictor and themotion vector of the current block may be indicated by signaling amotion vector difference.

The predictor 220 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply both intra prediction and inter prediction.This may be called combined inter and intra prediction (CIIP). Inaddition, the predictor may be based on an intra block copy (IBC)prediction mode or a palette mode for prediction of a block. The IBCprediction mode or palette mode may be used for content image/videocoding of a game or the like, for example, screen content coding (SCC).The IBC basically performs prediction in the current picture but may beperformed similarly to inter prediction in that a reference block isderived in the current picture. That is, the IBC may use at least one ofthe inter prediction techniques described in the present disclosure. Thepalette mode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The prediction signal generated by the predictor (including the interpredictor 221 and/or the intra predictor 222) may be used to generate areconstructed signal or to generate a residual signal. The transformer232 may generate transform coefficients by applying a transformtechnique to the residual signal. For example, the transform techniquemay include at least one of a discrete cosine transform (DCT), adiscrete sine transform (DST), a karhunen-loève transform (KLT), agraph-based transform (GBT), or a conditionally non-linear transform(CNT). Here, the GBT means transform obtained from a graph whenrelationship information between pixels is represented by the graph. TheCNT refers to transform generated based on a prediction signal generatedusing all previously reconstructed pixels. In addition, the transformprocess may be applied to square pixel blocks having the same size ormay be applied to blocks having a variable size rather than square.

The quantizer 233 may quantize the transform coefficients and transmitthem to the entropy encoder 240 and the entropy encoder 240 may encodethe quantized signal (information on the quantized transformcoefficients) and output a bitstream. The information on the quantizedtransform coefficients may be referred to as residual information. Thequantizer 233 may rearrange block type quantized transform coefficientsinto a one-dimensional vector form based on a coefficient scanning orderand generate information on the quantized transform coefficients basedon the quantized transform coefficients in the one-dimensional vectorform. Information on transform coefficients may be generated. Theentropy encoder 240 may perform various encoding methods such as, forexample, exponential Golomb, context-adaptive variable length coding(CAVLC), context-adaptive binary arithmetic coding (CABAC), and thelike. The entropy encoder 240 may encode information necessary forvideo/image reconstruction other than quantized transform coefficients(ex. values of syntax elements, etc.) together or separately. Encodedinformation (ex. encoded video/image information) may be transmitted orstored in units of NALs (network abstraction layer) in the form of abitstream. The video/image information may further include informationon various parameter sets such as an adaptation parameter set (APS), apicture parameter set (PPS), a sequence parameter set (SPS), or a videoparameter set (VPS). In addition, the video/image information mayfurther include general constraint information. In the presentdisclosure, information and/or syntax elements transmitted/signaled fromthe encoding apparatus to the decoding apparatus may be included invideo/picture information. The video/image information may be encodedthrough the above-described encoding procedure and included in thebitstream. The bitstream may be transmitted over a network or may bestored in a digital storage medium. The network may include abroadcasting network and/or a communication network, and the digitalstorage medium may include various storage media such as USB, SD, CD,DVD, Blu-ray, HDD, SSD, and the like. A transmitter (not shown)transmitting a signal output from the entropy encoder 240 and/or astorage unit (not shown) storing the signal may be included asinternal/external element of the encoding apparatus 200, andalternatively, the transmitter may be included in the entropy encoder240.

The quantized transform coefficients output from the quantizer 233 maybe used to generate a prediction signal. For example, the residualsignal (residual block or residual samples) may be reconstructed byapplying dequantization and inverse transform to the quantized transformcoefficients through the dequantizer 234 and the inverse transformer235. The adder 250 adds the reconstructed residual signal to theprediction signal output from the inter predictor 221 or the intrapredictor 222 to generate a reconstructed signal (reconstructed picture,reconstructed block, reconstructed sample array). If there is noresidual for the block to be processed, such as a case where the skipmode is applied, the predicted block may be used as the reconstructedblock. The adder 250 may be called a reconstructor or a reconstructedblock generator. The generated reconstructed signal may be used forintra prediction of a next block to be processed in the current pictureand may be used for inter prediction of a next picture through filteringas described below.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied duringpicture encoding and/or reconstruction.

The filter 260 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter260 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 270, specifically, a DPB of thememory 270. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like. The filter 260 may generate variousinformation related to the filtering and transmit the generatedinformation to the entropy encoder 240 as described later in thedescription of each filtering method. The information related to thefiltering may be encoded by the entropy encoder 240 and output in theform of a bitstream.

The modified reconstructed picture transmitted to the memory 270 may beused as the reference picture in the inter predictor 221. When the interprediction is applied through the encoding apparatus, predictionmismatch between the encoding apparatus 200 and the decoding apparatus300 may be avoided and encoding efficiency can be improved.

The DPB of the memory 270 DPB may store the modified reconstructedpicture for use as a reference picture in the inter predictor 221. Thememory 270 may store the motion information of the block from which themotion information in the current picture is derived (or encoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 221 and used as the motion information of thespatial neighboring block or the motion information of the temporalneighboring block. The memory 270 may store reconstructed samples ofreconstructed blocks in the current picture and may transfer thereconstructed samples to the intra predictor 222.

FIG. 3 is a schematic diagram illustrating a configuration of avideo/image decoding apparatus to which the embodiment(s) of the presentdisclosure may be applied.

Referring to FIG. 3 , the decoding apparatus 300 may include an entropydecoder 310, a residual processor 320, a predictor 330, an adder 340, afilter 350, a memory 360. The predictor 330 may include an interpredictor 331 and an intra predictor 332. The residual processor 320 mayinclude a dequantizer 321 and an inverse transformer 321. The entropydecoder 310, the residual processor 320, the predictor 330, the adder340, and the filter 350 may be configured by a hardware component (ex. Adecoder chipset or a processor) according to an embodiment. In addition,the memory 360 may include a decoded picture buffer (DPB) or may beconfigured by a digital storage medium. The hardware component mayfurther include the memory 360 as an internal/external component.

When a bitstream including video/image information is input, thedecoding apparatus 300 may reconstruct an image corresponding to aprocess in which the video/image information is processed in theencoding apparatus of FIG. 2 . For example, the decoding apparatus 300may derive units/blocks based on block partition related informationobtained from the bitstream. The decoding apparatus 300 may performdecoding using a processor applied in the encoding apparatus. Thus, theprocessor of decoding may be a coding unit, for example, and the codingunit may be partitioned according to a quad tree structure, binary treestructure and/or ternary tree structure from the coding tree unit or thelargest coding unit. One or more transform units may be derived from thecoding unit. The reconstructed image signal decoded and output throughthe decoding apparatus 300 may be reproduced through a reproducingapparatus.

The decoding apparatus 300 may receive a signal output from the encodingapparatus of FIG. 2 in the form of a bitstream, and the received signalmay be decoded through the entropy decoder 310. For example, the entropydecoder 310 may parse the bitstream to derive information (ex.video/image information) necessary for image reconstruction (or picturereconstruction). The video/image information may further includeinformation on various parameter sets such as an adaptation parameterset (APS), a picture parameter set (PPS), a sequence parameter set(SPS), or a video parameter set (VPS). In addition, the video/imageinformation may further include general constraint information. Thedecoding apparatus may further decode picture based on the informationon the parameter set and/or the general constraint information.Signaled/received information and/or syntax elements described later inthe present disclosure may be decoded may decode the decoding procedureand obtained from the bitstream. For example, the entropy decoder 310decodes the information in the bitstream based on a coding method suchas exponential Golomb coding, CAVLC, or CABAC, and output syntaxelements required for image reconstruction and quantized values oftransform coefficients for residual. More specifically, the CABACentropy decoding method may receive a bin corresponding to each syntaxelement in the bitstream, determine a context model using a decodingtarget syntax element information, decoding information of a decodingtarget block or information of a symbol/bin decoded in a previous stage,and perform an arithmetic decoding on the bin by predicting aprobability of occurrence of a bin according to the determined contextmodel, and generate a symbol corresponding to the value of each syntaxelement. In this case, the CABAC entropy decoding method may update thecontext model by using the information of the decoded symbol/bin for acontext model of a next symbol/bin after determining the context model.The information related to the prediction among the information decodedby the entropy decoder 310 may be provided to the predictor (the interpredictor 332 and the intra predictor 331), and the residual value onwhich the entropy decoding was performed in the entropy decoder 310,that is, the quantized transform coefficients and related parameterinformation, may be input to the residual processor 320. The residualprocessor 320 may derive the residual signal (the residual block, theresidual samples, the residual sample array). In addition, informationon filtering among information decoded by the entropy decoder 310 may beprovided to the filter 350. Meanwhile, a receiver (not shown) forreceiving a signal output from the encoding apparatus may be furtherconfigured as an internal/external element of the decoding apparatus300, or the receiver may be a component of the entropy decoder 310.Meanwhile, the decoding apparatus according to the present disclosuremay be referred to as a video/image/picture decoding apparatus, and thedecoding apparatus may be classified into an information decoder(video/image/picture information decoder) and a sample decoder(video/image/picture sample decoder). The information decoder mayinclude the entropy decoder 310, and the sample decoder may include atleast one of the dequantizer 321, the inverse transformer 322, the adder340, the filter 350, the memory 360, the inter predictor 332, and theintra predictor 331.

The dequantizer 321 may dequantize the quantized transform coefficientsand output the transform coefficients. The dequantizer 321 may rearrangethe quantized transform coefficients in the form of a two-dimensionalblock form. In this case, the rearrangement may be performed based onthe coefficient scanning order performed in the encoding apparatus. Thedequantizer 321 may perform dequantization on the quantized transformcoefficients by using a quantization parameter (ex. quantization stepsize information) and obtain transform coefficients.

The inverse transformer 322 inversely transforms the transformcoefficients to obtain a residual signal (residual block, residualsample array).

The predictor may perform prediction on the current block and generate apredicted block including prediction samples for the current block. Thepredictor may determine whether intra prediction or inter prediction isapplied to the current block based on the information on the predictionoutput from the entropy decoder 310 and may determine a specificintra/inter prediction mode.

The predictor 320 may generate a prediction signal based on variousprediction methods described below. For example, the predictor may notonly apply intra prediction or inter prediction to predict one block butalso simultaneously apply intra prediction and inter prediction. Thismay be called combined inter and intra prediction (CIIP). In addition,the predictor may be based on an intra block copy (IBC) prediction modeor a palette mode for prediction of a block. The IBC prediction mode orpalette mode may be used for content image/video coding of a game or thelike, for example, screen content coding (SCC). The IBC basicallyperforms prediction in the current picture but may be performedsimilarly to inter prediction in that a reference block is derived inthe current picture. That is, the IBC may use at least one of the interprediction techniques described in the present disclosure. The palettemode may be considered as an example of intra coding or intraprediction. When the palette mode is applied, a sample value within apicture may be signaled based on information on the palette table andthe palette index.

The intra predictor 331 may predict the current block by referring tothe samples in the current picture. The referred samples may be locatedin the neighborhood of the current block or may be located apartaccording to the prediction mode. In the intra prediction, predictionmodes may include a plurality of non-directional modes and a pluralityof directional modes. The intra predictor 331 may determine theprediction mode applied to the current block by using a prediction modeapplied to a neighboring block.

The inter predictor 332 may derive a predicted block for the currentblock based on a reference block (reference sample array) specified by amotion vector on a reference picture. In this case, in order to reducethe amount of motion information transmitted in the inter predictionmode, motion information may be predicted in units of blocks,sub-blocks, or samples based on correlation of motion informationbetween the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index.The motion information may further include inter prediction direction(L0 prediction, L1 prediction, Bi prediction, etc.) information. In thecase of inter prediction, the neighboring block may include a spatialneighboring block present in the current picture and a temporalneighboring block present in the reference picture. For example, theinter predictor 332 may configure a motion information candidate listbased on neighboring blocks and derive a motion vector of the currentblock and/or a reference picture index based on the received candidateselection information. Inter prediction may be performed based onvarious prediction modes, and the information on the prediction mayinclude information indicating a mode of inter prediction for thecurrent block.

The adder 340 may generate a reconstructed signal (reconstructedpicture, reconstructed block, reconstructed sample array) by adding theobtained residual signal to the prediction signal (predicted block,predicted sample array) output from the predictor (including the interpredictor 332 and/or the intra predictor 331). If there is no residualfor the block to be processed, such as when the skip mode is applied,the predicted block may be used as the reconstructed block.

The adder 340 may be called reconstructor or a reconstructed blockgenerator. The generated reconstructed signal may be used for intraprediction of a next block to be processed in the current picture, maybe output through filtering as described below, or may be used for interprediction of a next picture.

Meanwhile, luma mapping with chroma scaling (LMCS) may be applied in thepicture decoding process.

The filter 350 may improve subjective/objective image quality byapplying filtering to the reconstructed signal. For example, the filter350 may generate a modified reconstructed picture by applying variousfiltering methods to the reconstructed picture and store the modifiedreconstructed picture in the memory 360, specifically, a DPB of thememory 360. The various filtering methods may include, for example,deblocking filtering, a sample adaptive offset, an adaptive loop filter,a bilateral filter, and the like.

The (modified) reconstructed picture stored in the DPB of the memory 360may be used as a reference picture in the inter predictor 332. Thememory 360 may store the motion information of the block from which themotion information in the current picture is derived (or decoded) and/orthe motion information of the blocks in the picture that have alreadybeen reconstructed. The stored motion information may be transmitted tothe inter predictor 260 so as to be utilized as the motion informationof the spatial neighboring block or the motion information of thetemporal neighboring block. The memory 360 may store reconstructedsamples of reconstructed blocks in the current picture and transfer thereconstructed samples to the intra predictor 331.

In the present document, the embodiments described in the filter 260,the inter predictor 221, and the intra predictor 222 of the encodingapparatus 200 may be the same as or respectively applied to correspondto the filter 350, the inter predictor 332, and the intra predictor 331of the decoding apparatus 300. The same may also apply to the unit 332and the intra predictor 331.

As described above, in video coding, prediction is performed to increasecompression efficiency. Through this, it is possible to generate apredicted block including prediction samples for a current block, whichis a block to be coded. Here, the predicted block includes predictionsamples in a spatial domain (or pixel domain). The predicted block isderived equally from the encoding device and the decoding device, andthe encoding device decodes information (residual information) on theresidual between the original block and the predicted block, not theoriginal sample value of the original block itself. By signaling to thedevice, image coding efficiency can be increased. The decoding apparatusmay derive a residual block including residual samples based on theresidual information, and generate a reconstructed block includingreconstructed samples by summing the residual block and the predictedblock, and generate a reconstructed picture including reconstructedblocks.

The residual information may be generated through transformation andquantization processes. For example, the encoding apparatus may derive aresidual block between the original block and the predicted block, andperform a transform process on residual samples (residual sample array)included in the residual block to derive transform coefficients, andthen, by performing a quantization process on the transformcoefficients, derive quantized transform coefficients to signal theresidual related information to the decoding apparatus (via abitstream). Here, the residual information may include locationinformation, a transform technique, a transform kernel, and aquantization parameter, value information of the quantized transformcoefficients etc. The decoding apparatus may performdequantization/inverse transformation process based on the residualinformation and derive residual samples (or residual blocks). Thedecoding apparatus may generate a reconstructed picture based on thepredicted block and the residual block. The encoding apparatus may alsodequantize/inverse transform the quantized transform coefficients forreference for inter prediction of a later picture to derive a residualblock, and generate a reconstructed picture based thereon.

In the present document, at least one of quantization/dequantizationand/or transform/inverse transform may be omitted. When thequantization/dequantization is omitted, the quantized transformcoefficient may be referred to as a transform coefficient. When thetransform/inverse transform is omitted, the transform coefficients maybe called coefficients or residual coefficients, or may still be calledtransform coefficients for uniformity of expression.

In the present document, a quantized transform coefficient and atransform coefficient may be referred to as a transform coefficient anda scaled transform coefficient, respectively. In this case, the residualinformation may include information on transform coefficient(s), and theinformation on the transform coefficient(s) may be signaled throughresidual coding syntax. Transform coefficients may be derived based onthe residual information (or information on the transformcoefficient(s)), and scaled transform coefficients may be derivedthrough inverse transform (scaling) on the transform coefficients.Residual samples may be derived based on an inverse transform(transform) of the scaled transform coefficients. This may beapplied/expressed in other parts of the present document as well.

Intra prediction may refer to prediction that generates predictionsamples for the current block based on reference samples in a picture towhich the current block belongs (hereinafter, referred to as a currentpicture). When intra prediction is applied to the current block,neighboring reference samples to be used for intra prediction of thecurrent block may be derived. The neighboring reference samples of thecurrent block may include samples adjacent to the left boundary of thecurrent block having a size of nW×nH and a total of 2×nH samplesneighboring the bottom-left, samples adjacent to the top boundary of thecurrent block and a total of 2×nW samples neighboring the top-right, andone sample neighboring the top-left of the current block. Alternatively,the neighboring reference samples of the current block may include aplurality of upper neighboring samples and a plurality of leftneighboring samples. In addition, the neighboring reference samples ofthe current block may include a total of nH samples adjacent to theright boundary of the current block having a size of nW×nH, a total ofnW samples adjacent to the bottom boundary of the current block, and onesample neighboring (bottom-right) neighboring bottom-right of thecurrent block.

However, some of the neighboring reference samples of the current blockmay not be decoded yet or available. In this case, the decoder mayconfigure the neighboring reference samples to use for prediction bysubstituting the samples that are not available with the availablesamples. Alternatively, neighboring reference samples to be used forprediction may be configured through interpolation of the availablesamples.

When the neighboring reference samples are derived, (i) the predictionsample may be derived based on the average or interpolation ofneighboring reference samples of the current block, and (ii) theprediction sample may be derived based on the reference sample presentin a specific (prediction) direction for the prediction sample among theperiphery reference samples of the current block. The case of (i) may becalled non-directional mode or non-angular mode and the case of (ii) maybe called directional mode or angular mode.

Furthermore, the prediction sample may also be generated throughinterpolation between the second neighboring sample and the firstneighboring sample located in a direction opposite to the predictiondirection of the intra prediction mode of the current block based on theprediction sample of the current block among the neighboring referencesamples. The above case may be referred to as linear interpolation intraprediction (LIP). In addition, chroma prediction samples may begenerated based on luma samples using a linear model. This case may becalled LM mode.

In addition, a temporary prediction sample of the current block may bederived based on filtered neighboring reference samples, and at leastone reference sample derived according to the intra prediction modeamong the existing neighboring reference samples, that is, unfilteredneighboring reference samples, and the temporary prediction sample maybe weighted-summed to derive the prediction sample of the current block.The above case may be referred to as position dependent intra prediction(PDPC).

In addition, a reference sample line having the highest predictionaccuracy among the neighboring multi-reference sample lines of thecurrent block may be selected to derive the prediction sample by usingthe reference sample located in the prediction direction on thecorresponding line, and then the reference sample line used herein maybe indicated (signaled) to the decoding apparatus, thereby performingintra-prediction encoding. The above case may be referred to asmulti-reference line (MRL) intra prediction or MRL based intraprediction.

In addition, intra prediction may be performed based on the same intraprediction mode by dividing the current block into vertical orhorizontal subpartitions, and neighboring reference samples may bederived and used in the subpartition unit. That is, in this case, theintra prediction mode for the current block is equally applied to thesubpartitions, and the intra prediction performance may be improved insome cases by deriving and using the neighboring reference samples inthe subpartition unit. Such a prediction method may be called intrasubpartitions (ISP) or ISP based intra prediction.

The above-described intra prediction methods may be called an intraprediction type separately from the intra prediction mode. The intraprediction type may be called in various terms such as an intraprediction technique or an additional intra prediction mode. Forexample, the intra prediction type (or additional intra prediction mode)may include at least one of the above-described LIP, PDPC, MRL, and ISP.A general intra prediction method except for the specific intraprediction type such as LIP, PDPC, MRL, or ISP may be called a normalintra prediction type. The normal intra prediction type may be generallyapplied when the specific intra prediction type is not applied, andprediction may be performed based on the intra prediction mode describedabove. Meanwhile, post-filtering may be performed on the predictedsample derived as needed.

Specifically, the intra prediction procedure may include an intraprediction mode/type determination step, a neighboring reference samplederivation step, and an intra prediction mode/type based predictionsample derivation step. In addition, a post-filtering step may beperformed on the predicted sample derived as needed.

When intra prediction is applied, the intra prediction mode applied tothe current block may be determined using the intra prediction mode ofthe neighboring block. For example, the decoding apparatus may selectone of most probable mode (mpm) candidates of an mpm list derived basedon the intra prediction mode of the neighboring block (ex. left and/orupper neighboring blocks) of the current block based on the received mpmindex and select one of the other remaining intro prediction modes notincluded in the mpm candidates (and planar mode) based on the remainingintra prediction mode information. The mpm list may be configured toinclude or not include a planar mode as a candidate. For example, if thempm list includes the planar mode as a candidate, the mpm list may havesix candidates. If the mpm list does not include the planar mode as acandidate, the mpm list may have three candidates. When the mpm listdoes not include the planar mode as a candidate, a not planar flag (ex.intra_luma_not_planar_flag) indicating whether an intra prediction modeof the current block is not the planar mode may be signaled. Forexample, the mpm flag may be signaled first, and the mpm index and notplanar flag may be signaled when the value of the mpm flag is 1. Inaddition, the mpm index may be signaled when the value of the not planarflag is 1. Here, the mpm list is configured not to include the planarmode as a candidate does not is to signal the not planar flag first tocheck whether it is the planar mode first because the planar mode isalways considered as mpm.

For example, whether the intra prediction mode applied to the currentblock is in mpm candidates (and planar mode) or in remaining mode may beindicated based on the mpm flag (ex. Intra_luma_mpm_flag). A value 1 ofthe mpm flag may indicate that the intra prediction mode for the currentblock is within mpm candidates (and planar mode), and a value 0 of thempm flag may indicate that the intra prediction mode for the currentblock is not in the mpm candidates (and planar mode). The value 0 of thenot planar flag (ex. Intra_luma_not_planar_flag) may indicate that theintra prediction mode for the current block is planar mode, and thevalue 1 of the not planar flag value may indicate that the intraprediction mode for the current block is not the planar mode. The mpmindex may be signaled in the form of an mpm_idx or intra_luma_mpm_idxsyntax element, and the remaining intra prediction mode information maybe signaled in the form of a rem_intra_luma_pred_mode orintra_luma_mpm_remainder syntax element. For example, the remainingintra prediction mode information may index remaining intra predictionmodes not included in the mpm candidates (and planar mode) among allintra prediction modes in order of prediction mode number to indicateone of them. The intra prediction mode may be an intra prediction modefor a luma component (sample). Hereinafter, intra prediction modeinformation may include at least one of the mpm flag (ex.Intra_luma_mpm_flag), the not planar flag (ex.Intra_luma_not_planar_flag), the mpm index (ex. mpm_idx orintra_luma_mpm_idx), and the remaining intra prediction mode information(rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the presentdocument, the MPM list may be referred to in various terms such as MPMcandidate list and candModeList. When MIP is applied to the currentblock, a separate mpm flag (ex. intra_mip_mpm_flag), an mpm index (ex.intra_mip_mpm_idx), and remaining intra prediction mode information (ex.intra_mip_mpm_remainder) for MIP may be signaled and the not planar flagis not signaled.

In other words, in general, when block splitting is performed on animage, a current block and a neighboring block to be coded have similarimage characteristics. Therefore, the current block and the neighboringblock have a high probability of having the same or similar intraprediction mode. Thus, the encoder may use the intra prediction mode ofthe neighboring block to encode the intra prediction mode of the currentblock.

For example, the encoder/decoder may configure a list of most probablemodes (MPM) for the current block. The MPM list may also be referred toas an MPM candidate list. Herein, the MPM may refer to a mode used toimprove coding efficiency in consideration of similarity between thecurrent block and neighboring block in intra prediction mode coding. Asdescribed above, the MPM list may be configured to include the planarmode or may be configured to exclude the planar mode. For example, whenthe MPM list includes the planar mode, the number of candidates in theMPM list may be 6. And, if the MPM list does not include the planarmode, the number of candidates in the MPM list may be 5.

The encoder/decoder may configure an MPM list including 5 or 6 MPMs.

In order to configure the MPM list, three types of modes can beconsidered: default intra modes, neighbor intra modes, and the derivedintra modes.

For the neighboring intra modes, two neighboring blocks, i.e., a leftneighboring block and an upper neighboring block, may be considered.

As described above, if the MPM list is configured not to include theplanar mode, the planar mode is excluded from the list, and the numberof MPM list candidates may be set to 5.

In addition, the non-directional mode (or non-angular mode) among theintra prediction modes may include a DC mode based on the average ofneighboring reference samples of the current block or a planar modebased on interpolation.

When inter prediction is applied, the predictor of the encodingapparatus/decoding apparatus may derive a prediction sample byperforming inter prediction in units of blocks. Inter prediction may bea prediction derived in a manner that is dependent on data elements (ex.sample values or motion information) of picture(s) other than thecurrent picture. When inter prediction is applied to the current block,a predicted block (prediction sample array) for the current block may bederived based on a reference block (reference sample array) specified bya motion vector on the reference picture indicated by the referencepicture index. Here, in order to reduce the amount of motion informationtransmitted in the inter prediction mode, the motion information of thecurrent block may be predicted in units of blocks, subblocks, or samplesbased on correlation of motion information between the neighboring blockand the current block. The motion information may include a motionvector and a reference picture index. The motion information may furtherinclude inter prediction type (L0 prediction, L1 prediction, Biprediction, etc.) information. In the case of inter prediction, theneighboring block may include a spatial neighboring block present in thecurrent picture and a temporal neighboring block present in thereference picture. The reference picture including the reference blockand the reference picture including the temporal neighboring block maybe the same or different. The temporal neighboring block may be called acollocated reference block, a co-located CU (colCU), and the like, andthe reference picture including the temporal neighboring block may becalled a collocated picture (colPic). For example, a motion informationcandidate list may be configured based on neighboring blocks of thecurrent block, and flag or index information indicating which candidateis selected (used) may be signaled to derive a motion vector and/or areference picture index of the current block. Inter prediction may beperformed based on various prediction modes. For example, in the case ofa skip mode and a merge mode, the motion information of the currentblock may be the same as motion information of the neighboring block. Inthe skip mode, unlike the merge mode, the residual signal may not betransmitted. In the case of the motion vector prediction (MVP) mode, themotion vector of the selected neighboring block may be used as a motionvector predictor and the motion vector of the current block may besignaled. In this case, the motion vector of the current block may bederived using the sum of the motion vector predictor and the motionvector difference.

The motion information may include L0 motion information and/or L1motion information according to an inter prediction type (L0 prediction,L1 prediction, Bi prediction, etc.). The motion vector in the L0direction may be referred to as an L0 motion vector or MVL0, and themotion vector in the L1 direction may be referred to as an L1 motionvector or MVL1. Prediction based on the L0 motion vector may be calledL0 prediction, prediction based on the L1 motion vector may be called L1prediction, and prediction based on both the L0 motion vector and the L1motion vector may be called bi-prediction. Here, the L0 motion vectormay indicate a motion vector associated with the reference picture listL0 (L0), and the L1 motion vector may indicate a motion vectorassociated with the reference picture list L1 (L1). The referencepicture list L0 may include pictures that are earlier in output orderthan the current picture as reference pictures, and the referencepicture list L1 may include pictures that are later in the output orderthan the current picture. The previous pictures may be called forward(reference) pictures, and the subsequent pictures may be called reverse(reference) pictures. The reference picture list L0 may further includepictures that are later in the output order than the current picture asreference pictures. In this case, the previous pictures may be indexedfirst in the reference picture list L0 and the subsequent pictures maybe indexed later. The reference picture list L1 may further includeprevious pictures in the output order than the current picture asreference pictures. In this case, the subsequent pictures may be indexedfirst in the reference picture list 1 and the previous pictures may beindexed later. The output order may correspond to picture order count(POC) order.

A video/image encoding procedure based on inter prediction may include,for example, the following.

FIG. 4 shows an example of a video/image encoding method based on interprediction.

The encoding apparatus performs inter prediction on the current block(S400). The encoding apparatus may derive inter prediction mode andmotion information of the current block and generate prediction samplesof the current block. Here, the inter prediction mode determination, themotion information derivation, and the prediction samples generationprocedure may be performed simultaneously, or one procedure may beperformed before the other. For example, the inter predictor of theencoding apparatus may include a prediction mode determiner, a motioninformation deriver, and a prediction sample deriver. The predictionmode determiner may determine a prediction mode for the current block,the motion information deriver may derive motion information of thecurrent block, and prediction sample deriver may derive motion samplesof the current block. For example, the inter predictor of the encodingapparatus may search for a block similar to the current block in apredetermined region (search region) of reference pictures throughmotion estimation and derive a reference block whose difference to thecurrent block is a minimum or a predetermined reference or less. Basedon this, the inter predictor may derive a reference picture indexindicating a reference picture in which the reference block is locatedand derive a motion vector based on a position difference between thereference block and the current block. The encoding apparatus maydetermine a mode applied to the current block among various predictionmodes. The encoding apparatus may compare RD costs for the variousprediction modes and determine an optimal prediction mode for thecurrent block.

For example, when a skip mode or a merge mode is applied to the currentblock, the encoding apparatus may configure a merge candidate list to bedescribed later and derive a reference block whose difference to thecurrent block is minimum or a predetermined reference or less amongreference blocks indicated by the merge candidates included in the mergecandidate list. In this case, a merge candidate associated with thederived reference block may be selected, and merge index informationindicating the selected merge candidate may be generated and signaled tothe decoding apparatus. The motion information of the current block maybe derived using the motion information of the selected merge candidate.

As another example, when the (A)MVP mode is applied to the currentblock, the encoding apparatus may configure a (A)MVP candidate list tobe described later and use a motion vector of an mvp candidate selectedfrom among the mvp (motion vector predictor) candidates included in the(A)MVP candidate list, as mvp of the current block. In this case, forexample, a motion vector indicating the reference block derived by theabove-described motion estimation may be used as the motion vector ofthe current block, and an mvp candidate having a motion vector whosedifference to the motion vector of the current block, among the mvpcandidates, is smallest may be the selected mvp candidate. A motionvector difference (MVP) which is a difference from which the mvp wassubtracted may be derived from the motion vector of the current block.In this case, the information on the MVD may be signaled to the decodingapparatus. In addition, when the (A)MVP mode is applied, the value ofthe reference picture index may be configured as reference picture indexinformation and separately signaled to the decoding apparatus.

The encoding apparatus may derive residual samples based on theprediction samples (S410). The encoding apparatus may derive theresidual samples by comparing the original samples of the current blockwith the prediction samples.

The encoding apparatus encodes image information including predictioninformation and residual information (S420). The encoding apparatus mayoutput the encoded image information in the form of a bitstream. Theprediction information may include prediction mode information (ex. skipflag, merge flag or mode index) and information on motion information asinformation related to the prediction procedure. The information on themotion information may include candidate selection information (ex.merge index, mvp flag or mvp index) that is information for deriving amotion vector. In addition, the information on the motion informationmay include the information on the MVD and/or reference picture indexinformation described above. The information on the motion informationmay include information indicating whether L0 prediction, L1 prediction,or bi prediction is applied. The residual information is information onthe residual samples. The residual information may include informationon quantized transform coefficients for the residual samples.

The output bitstream may be stored in a (digital) storage medium anddelivered to the decoding apparatus, or may be delivered to the decodingapparatus via a network.

Meanwhile, as described above, the encoding apparatus may generate areconstructed picture (including the reconstructed samples and thereconstructed block) based on the reference samples and the residualsamples. This is to derive the same prediction result in the encodingapparatus as that performed in the decoding apparatus and because codingefficiency may be increased. Therefore, the encoding apparatus may storea reconstructed picture (or reconstructed samples, a reconstructedblock) in the memory and use it as a reference picture for interprediction. As described above, the in-loop filtering procedure may befurther applied to the reconstructed picture.

A video/image decoding procedure based on inter prediction may include,for example, the following.

FIG. 5 shows an example of a video/image decoding method based on interprediction.

Referring to FIG. 5 , the decoding apparatus may perform an operationcorresponding to the operation performed in the encoding apparatus. Thedecoding apparatus may perform prediction on the current block based onthe received prediction information and derive prediction samples.

Specifically, the decoding apparatus may determine a prediction mode forthe current block based on the received prediction information (S500).The decoding apparatus may determine which inter prediction mode isapplied to the current block based on the prediction mode information inthe prediction information.

For example, the decoding apparatus may determine whether the merge modeis applied to the current block or whether (A)MVP mode is determinedbased on the merge flag. Alternatively, one of various inter predictionmode candidates may be selected based on the mode index. The interprediction mode candidates may include a skip mode, a merge mode, and/or(A)MVP mode, or may include various inter prediction modes describedbelow.

The decoding apparatus derives motion information of the current blockbased on the determined inter prediction mode (S510). For example, whena skip mode or a merge mode is applied to the current block, thedecoding apparatus may configure a merge candidate list to be describedlater, and select one of the merge candidates included in the mergecandidate list. The selection may be performed based on the aboveselection information (merge index). The motion information of thecurrent block may be derived using the motion information of theselected merge candidate. The motion information of the selected mergecandidate may be used as motion information of the current block.

As another example, when the (A)MVP mode is applied to the currentblock, the decoding apparatus may configure an (A)MVP candidate list tobe described later and use a motion vector of an mvp candidate selectedfrom the mvp candidates included in the (A)MVP candidate list as mvp ofthe current block. The selection may be performed based on theabove-described selection information (mvp flag or mvp index). In thiscase, the MVD of the current block may be derived based on theinformation on the MVD, and the motion vector of the current block maybe derived based on mvp and the MVD of the current block. In addition, areference picture index of the current block may be derived based on thereference picture index information. A picture indicated by thereference picture index in the reference picture list for the currentblock may be derived as a reference picture referred for interprediction of the current block.

Meanwhile, as described below, motion information of the current blockmay be derived without configuring a candidate list, and in this case,motion information of the current block may be derived according to aprocedure disclosed in a prediction mode to be described later. In thiscase, the configuration of the candidate list as described above may beomitted.

The decoding apparatus may generate prediction samples for the currentblock based on the motion information of the current block (S520). Inthis case, the reference picture may be derived based on the referencepicture index of the current block, and the prediction samples of thecurrent block may be derived using the samples of the reference blockindicated by the motion vector of the current block on the referencepicture. In this case, as described below, a prediction sample filteringprocedure may be further performed on all or some of the predictionsamples of the current block.

For example, the inter predictor of the decoding apparatus may include aprediction mode determiner, a motion information deriver, and aprediction sample deriver. The prediction mode for the current block maybe determined based on the prediction mode information received from theprediction mode determiner, motion information (motion vector and/orreference picture index, etc.) of the current block may be derived basedon the information on the motion information received from the motioninformation deriver, and the prediction sample deriver may derive theprediction samples of the current block.

The decoding apparatus generates residual samples for the current blockbased on the received residual information (S530). The decodingapparatus may generate reconstructed samples for the current block basedon the prediction samples and the residual samples and generate areconstructed picture based on the reconstructed samples (S540).Thereafter, the in-loop filtering procedure or the like may be furtherapplied as described above.

FIG. 6 exemplarily shows an inter prediction procedure.

Referring to FIG. 6 , as described above, the inter prediction proceduremay include determining an inter prediction mode, deriving motioninformation according to the determined prediction mode, and performingprediction based on the derived motion information (generation of aprediction sample). The inter prediction procedure may be performed bythe encoding apparatus and the decoding apparatus as described above. Inthis document, a coding apparatus may include an encoding apparatusand/or a decoding apparatus.

Referring to FIG. 6 , the coding apparatus determines an interprediction mode for the current block (S600). Various inter predictionmodes may be used for prediction of the current block in the picture.For example, various modes, such as a merge mode, a skip mode, a motionvector prediction (MVP) mode, an affine mode, a subblock merge mode, anda merge with MVD (MMVD) mode, and the like may be used. A decoder sidemotion vector refinement (DMVR) mode, an adaptive motion vectorresolution (AMVR) mode, a bi-prediction with CU-level weight (BCW), abi-directional optical flow (BDOF), and the like may also be used asadditional modes additionally or instead. The affine mode may be calledan affine motion prediction mode. The MVP mode may be referred to asadvanced motion vector prediction (AMVP) mode. In this document, somemodes and/or motion information candidates derived by some modes may beincluded as one of motion information candidates of other modes. Forexample, an HMVP candidate may be added as a merge candidate in themerge/skip mode or may be added as an MVP candidate in the MVP mode.When the HMVP candidate is used as a motion information candidate in themerge mode or the skip mode, the HMVP candidate may be referred to as anHMVP merge candidate.

Prediction mode information indicating the inter prediction mode of thecurrent block may be signaled from the encoding apparatus to thedecoding apparatus. The prediction mode information may be included inthe bitstream and received by the decoding apparatus. The predictionmode information may include index information indicating one of aplurality of candidate modes. Alternatively, the inter prediction modemay be indicated through hierarchical signaling of flag information. Inthis case, the prediction mode information may include one or moreflags. For example, a skip flag may be signaled to indicate whether askip mode is applied, and if the skip mode is not applied, a merge flagmay be signaled to indicate whether a merge mode is applied, and if themerge mode is not applied, it is indicated to apply an MVP mode or aflag for additional classification may be further signaled. The affinemode may be signaled in an independent mode or may be signaled in a modedependent on the merge mode or the MVP mode. For example, the affinemode may include an affine merge mode and an affine MVP mode.

Meanwhile, information indicating whether the list0 (L0) prediction, thelist1 (L1) prediction, or the bi-prediction described above is used inthe current block (current coding unit) may be signaled in the currentblock. The information may be referred to as motion prediction directioninformation, inter prediction direction information or inter predictionindication information, and may be configured/encoded/signaled in theform of, for example, an inter_pred_idc syntax element. That is, theinter_pred_idc syntax element may indicate whether the aforementionedlist0 (L0) prediction, list1 (L1) prediction, or bi-prediction is usedfor the current block (current coding unit). In this document, for theconvenience of description, the inter prediction type (L0 prediction, L1prediction, or BI prediction) indicated by the inter_pred_idc syntaxelement may be indicated as a motion prediction direction. L0 predictionmay be represented as pred_L0, L1 prediction as pred_L1, andbi-prediction as pred_BI. For example, the following prediction type maybe indicated according to the value of the inter_pred_idc syntaxelement.

TABLE 1 Value of Name of inter_pred_idc inter_pred_idc ( cbWidth +cbHeight ) != 8 ( cbWidth + cbHeight ) = = 8 0 PRED_L0 PRED_L0 1 PRED_L1PRED_L1 2 PRED_BI —

As described above, one picture may include one or more slices. Theslice may have one of slice types including intra (I) slice, predictive(P) slice, and bi-predictive (B) slice. The slice type may be indicatedbased on slice type information. For blocks in an I slice, interprediction may not be used for prediction and only intra prediction maybe used. Of course, even in this case, the original sample value may becoded and signaled without prediction. Intra prediction or interprediction may be used for blocks in a P slice, and only uni predictionmay be used when inter prediction is used. Meanwhile, intra predictionor inter prediction may be used for blocks in a B slice, and up to biprediction may be used when inter prediction is used.

L0 and L1 may include reference pictures that are previouslyencoded/decoded prior to the current picture. For example, L0 mayinclude reference pictures before and/or after the current picture inPOC order, and L1 may include reference pictures after and/or before thecurrent picture in POC order. In this case, L0 may be assigned a lowerreference picture index relative to previous reference pictures in thePOC order than the current reference pictures, and L1 may be assigned alower reference picture index relative to previous reference pictures inthe POC order than the current picture. In the case of B slice,bi-prediction may be applied, and in this case, unidirectionalbi-prediction may be applied or bidirectional bi-prediction may beapplied. The bidirectional bi-prediction may be called truebi-prediction.

The following table shows syntax for a coding unit according to anembodiment of this document.

TABLE 2 Descriptor coding_unit( x0, y0, cbWidth, cbHeight, treeType ) { if( slice_type != I ∥ sps_ibc_enabled_flag) {   if( treeType !=DUAL_TREE_CHROMA &&    !( cbWidth = = 4 && cbHeight = = 4 &&!sps_ibc_enabled_flag ) )     cu_skip_flag[ x0 ][ y0 ] ae (v)   if(cu_skip_flag[ x0 ][ y0 ] ==0 && slice_type != I    && ! ( cbWidth = = 4&& cbHeight = = 4 ) )     pred_mode_flag ae (v)   if( ( ( slice_type = =I && cu_skip_flag[ x0 ][ y0 ] = =0 ) ∥     ( slice_type != I && (CuPredMode[ x0 ][ y0 ] != MODE_INTRA ∥     ( cbWidth = = 4 && cbHeight == 4 && cu_skip_flag[ x0 ][ y0 ] = = 0 ) ) ) ) &&    sps_ibc_enabled_flag && ( cbWidth != 128 ∥ cbHeight != 128 ) )    pred_mode_ibc_flag ae (v)  }  if( CuPredMode[ x0 ][ y0 ] = =MODE_INTRA ) {   if( sps_pcm_enabled_flag &&     cbWidth >=MinIpcmCbSizeY && cbWidth <= MaxIpcmCbSizeY &&     cbHeight >=MinIpcmCbSizeY && cbHeight <= MaxIpcmCbSizeY )     pcm_flag[ x0 ][ y0 ]ae (v)   if( pcm_flag[ x0 ][ y0 ] ) {     while( !byte_aligned( ) )     pcm_alignment_zero_bit  f (1)     pcm_sample( cbWidth, cbHeight,treeType )   } else {     if( treeType = = SINGLE_TREE ∥ treeType = =DUAL_TREE_LUMA ) {      if( cbWidth <= 32 && cbHeight <= 32 )      intra_bdpcm_flag[ x0 ][ y0 ] ae (v)      if( intra_bdpcm_flag[ x0][ y0 ] )       intra_bdpcm_dir_flag[ x0 ][ y0 ] ae (v)      else {      if( sps_mip_enabled_flag &&        ( Abs( Log2( cbWidth) - Log2(cbHeight)) <= 2 ) &&         cbWidth <= MaxTbSizeY && cbHeight <=MaxTbSizeY )        intra_mip_flag[ x0 ][ y0 ] ae (v)       if(intra_mip_flag[ x0 ][ y0 ] ) {         intra_mip_mpm_flag[ x0 ][ y0 ] ae(v)        if( intra_mip_mpm_flag[ x0 ][ y0 ] )        intra_mip_mpm_idx[ x0 ][ y0 ] ae (v)        else        intra_mip_mpm_remainder[ x0 ][ y0 ] ae (v)       } else {       if( sps_mrl_enabled_flag && ( ( y0 % CtbSizeY ) > 0 ) )        intra_luma_ref_idx[ x0 ][ y0 ] ae (v)       if (sps_isp_enabled_flag && intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&       (cb Width <= MaxTbSizeY && cbHeight <= MaxTbSizeY ) &&        (cbWidth * cbHeight > MinTbSizeY * MinTbSizeY ) )       intra_subpartitions_mode_flag[ x0 ][ y0 ] ae (v)       if(intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 1 &&        cbWidth <=MaxTbSizeY && cbHeight <= MaxTbSizeY )       intra_subpartitions_split_flag[ x0 ][ y0 ] ae (v)       if(intra_luma_ref_idx[ x0 ][ y0 ] = = 0 &&       intra_subpartitions_mode_flag[ x0 ][ y0 ] = = 0 )       intra_luma_mpm_flag[ x0 ][ y0 ] ae (v)       if(intra_luma_mpm_flag[ x0 ][ y0 ] ) {        if( intra_luma_ref_idx[ x0 ][y0 ] = = 0 )         intra_luma_not_planar_flag[ x0 ][ y0 ] ae (v)       if( intra_luma_not_planar_flag[ x0 ][ y0 ] )        intra_luma_mpm_idx[ x0 ][ y0 ] ae (v)       } else       intra_luma_mpm_remainder[ x0 ][ y0 ] ae (v)      }     }    }   if( treeType = = SINGLE_TREE ∥ treeType = = DUAL_TREE_CHROMA )    intra_chroma_pred_mode[ x0 ][ y0 ] ae (v)   }  } else if( treeType!= DUAL_TREE_CHROMA ) { /* MODE_ INTER or MODE_IBC */   if(cu_skip_flag[ x0 ][ y0 ] = = 0 )    general_merge_flag[ x0 ][ y0 ] ae(v)   if( general_merge_flag[ x0 ][ y0 ] ) {    merge_data( x0, y0,cbWidth, cbHeight )   } else if ( CuPredMode [ x0 ][ y0 ] = = MODE_IBC ){    mvd_coding( x0, y0, 0, 0 )    mvp_l0_flag[ x0 ][ y0 ] ae (v)    if(sps_amvr_enabled_flag &&     ( MvdL0[ x0 ][ y0 ][ 0 ] != 0 ∥ MvdL0[ x0][ y0 ][ 1 ] != 0 ) ) {     amvr_precision_flag[ x0 ][ y0 ] ae (v)    }  } else {    if( slice_type = = B )     inter_pred_idc[ x0 ][ y0 ] ae(v)    if( sps_affine_enabled_flag && cbWidth >= 16 && cbHeight >= 16 ){     inter_affine_flag[ x0 ][ y0 ] ae (v)     if( sps_affine_type_flag&& inter_affine_flag[ x0 ][ y0 ] )      cu_affine_type_flag[ x0 ][ y0 ]ae (v)    }    if( sps_smvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] == PRED_BI &&     !inter_affine_flag[ x0 ][ y0 ] && RefIdxSymL0 > −1 &&RefIdxSvmL1 > −1 )     sym_mvd_flag[ x0 ][ y0 ] ae (v)    if(inter_pred_idc[ x0 ][ y0 ] != PRED_L1 ) {     if( NumRefIdxActive [ 0] > 1 && ! sym_mvd_flag[ x0 ][ y0 ] )       ref_idx_l0[ x0 ][ y0 ] ae(v)      mvd_coding( x0, y0, 0, 0 )      if( MotionModelIdc[ x0 ][ y0] > 0 )       mvd_coding( x0, y0, 0, 1 )      if(MotionModelIdc[ x0 ][y0 ] > 1 )       mvd_coding( x0, y0, 0, 2 )      mvp_l0_flag[ x0 ][ y0 ]ae (v)     } else {      MvdL0[ x0 ][ y0 ][ 0 ] = 0      MvdLn[ x0 ][ y0][ 1 ] = 0     }     if( inter_pred_idc[ x0 ][ y0 ] != PRED_L0 ) {     if( NumRefIdxActive[ 1 ] > 1 && !sym_mvd_flag[ x0 ][ y0 ] )      ref_idx_l1[ x0 ][ y0 ] ae (v)      if( mvd_l1_zero_flag &&inter_pred_idc[ x0 ][ y0 ] = = PRED_BI ) {       MvdL1[ x0 ][ y0 ][ 0 ]= 0       MvdL1[ x0 ][ y0 ][ 1 ] = 0       MvdCpL1[ x0 ][ y0 ][ 0 ][ 0 ]= 0       MvdCpL1[ x0 ][ y0 ][ 0 ][ 1 ] = 0       MvdCpL1[ x0 ][ y0 ][ 1][ 0 ] = 0       MvdCpL1[ x0 ][ y0 ][ 1 ][ 1 ] = 0       MvdCpL1[ x0 ][y0 ][ 2 ][ 0 ] = 0       MvdCpL1[ x0 ][ y0 ][ 2 ][ 1 ] = 0      } else {      if( sym_mvd_flag[ x0 ][ y0 ] ) {        MvdL1[ x0 ][ y0 ][ 0 ] =−MvdL0[ x0 ][ y0 ][ 0 ]        MvdL1[ x0 ][ y0 ][ 1 ] = −MvdL0[ x0 ][ y0][ 1 ]       } else        mvd_coding( x0, y0, 1, 0 )       if(MotionModelIdc [ x0 ][ y0 ] > 0 )        mvd_coding( x0, y0 1, 1 )      if( MotionModelIdc[ x0 ][ y0 ] > 1 )        mvd_coding( x0, y0, 1,2 )       mvp_l1_flag[ x0 ][ y0 ] ae (v)      }     } else {      MvdL1[x0 ][ y0 ][ 0 ] = 0      MvdL1[ x0 ][ y0 ][ 1 ] = 0     }    if( (sps_amvr_enabled_flag && inter_affine_flag[ x0 ][ y0 ] = = 0 &&        (MvdL0[ x0 ][ y0 ] != 0 ∥ MvdL0[ x0 ][ y0 ][ 1 ] != 0 ∥        MvdL1[ x0][ y0 ] != 0 ∥ MvdL1[ x0 ][ y0 ][ 1 ] != 0 ) ) ∥       (sps_affine_amvr_enabled_flag && inter_affine_flag[ x0 ][ y0 ] = = 1 &&       ( MvdCpL0[ x0 ][ y0 ][ 0 ][ 0 ] !=0 ∥ MvdCpL0[ x0 ][ y0 ][ 0 ][ 1] != 0 ∥         MvdCpL1[ x0 ][ y0 ][ 0 ][ 0 ] !=0 ∥ MvdCpL1[ x0 ][ y0][ 0 ][ 1 ] != 0 ∥         MvdCpL0[ x0 ][ y0 ][ 1 ][ 0 ] !=0 ∥ MvdCpL0[x0 ][ y0 ][ 1 ][ 1 ] != 0 ∥         MvdCpL1[ x0 ][ y0 ][ 1 ][ 0 ] !=0 ∥MvdCpL1[ x0 ][ y0 ][ 1 ][ 1 ] != 0 ∥         MvdCpL0[ x0 ][ y0 ][ 2 ][ 0] !=0 ∥ MvdCpL0[ x0 ][ y0 ][ 2 ][ 1 ] != 0 ∥         MvdCpL1[ x0 ][ y0][ 2 ][ 0 ] !=0 ∥ MvdCpL1[ x0 ][ y0 ][ 2 ][ 1 ] != 0 ) ) {     amvr_flag[ x0 ][ y0 ] ae (v)      if( amvr_flag[ x0 ][ y0 ] )       amvr_precision_flag[ x0 ][ y0 ] ae (v)    }     if(sps_bcw_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = = PRED_BI &&      luma_weight_l0_flag[ ref_idx_l0 [ x0 ][ y0 ] ] == 0 &&      luma_weight_l1_flag[ ref_idx_l1 [ x0 ][ y0 ] ] == 0 &&      chroma_weight_l0_flag[ ref_idx_l0 [ x0 ][ y0 ] ] = = 0 &&      chroma_weight_l1_flag[ ref_idx_l1 [ x0 ][ y0 ] ] = = 0 &&      cbWidth * cbHeight >= 256 )      bcw_idx[ x0 ][ y0 ] ae (v)   }  } if( !pcm_flag[ x0 ][ y0 ] ) {   if( CuPredMode[ x0 ][ y0 ] !=MODE_INTRA &&    general_merge_flag[ x0 ][ y0 ] = = 0 )    cu_cbf ae (v)  if( cu_cbf ) {    if( CuPredMode[ x0 ][ y0 ] = = MODE_INTER &&sps_sbt_enabled_flag &&     !ciip_flag[ x0 ][ y0 ] && MergeTriangleFlag[x0 ][ y0 ] ) {     if( cbWidth <= MaxSbtSize && cbHeight <= MaxSbtSize ){      allowSbtVerH = cbWidth >= 8      allowSbtVerQ = cbWidth >= 16     allowSbtHorH = cbHeight >= 8      allowSbtHorQ = cbHeight >= 16     if( allowSbtVerH ∥ allowSbtHorH ∥ allowSbtVerQ ∥ allowSbtHorQ )       cu_sbt_flag ae (v)     }     if( cu_sbt_flag ) {      if(allowSbtVerHallowSbtHorH) && (allow SbtVerQ ∥ allowSbtHorQ ) )       cu_sbt_quad_flag ae (v)      if( (cu_sbt_quad_flag &&allowSbtVerQ && allowSbtHorQ ) ∥         ( !cu_sbt_quad_flag &&allowSbtVerH && allowSbtHorH ) )        cu_sbt_horizontal_flag ae (v)     cu_sbt_pos_flag ae (v)     }    }    numSigCoeff = 0   numZeroOutSigCoeff = 0    transform_tree( x0, y0, cbWidth, cbHeight,treeType )    lfnstWidth = ( treeType = = DUAL_TREE_CHROMA ) ? cbWidth /SubWidthC          : cbWidth    lfnstHeight = (treeType = =DUAL_TREE_CHROMA ) ? cbHeight / SubHeightC          : cbHeight    if(Min( lfnstWidth, lfnstHeight) >= 4 && sps_lfnst_enabled_flag = = 1 &&    CuPredMode[ x0 ][ y0 ] = = MODE_INTRA &&    IntraSubPartitionsSplitType = = ISP_NO_SPLIT &&     !intra_mip_flag[x0 ][ y0 ] ) {     if( (numSigCoeff > ( ( treeType = = SINGLE_TREE ) ? 2: 1 ) ) &&       numZeroOutSigCoeff = = 0 )      lfnst_idx[ x0 ][ y0 ]ae (v)    }   }  } }

The coding apparatus derives motion information for the current block(S610). The motion information derivation may be derived based on theinter prediction mode.

Inter prediction may be performed using motion information of thecurrent block. The encoding apparatus may derive optimal motioninformation for the current block through a motion estimation procedure.For example, the encoding apparatus may search for a similar referenceblock having a high correlation in units of fractional pixels within apredetermined search range in the reference picture using the originalblock in the original picture for the current block, thereby derivingmotion information. Similarity of blocks may be derived based on adifference of phase based sample values. For example, the similarity ofthe blocks may be calculated based on the SAD between the current block(or template of the current block) and the reference block (or templateof the reference block). In this case, motion information may be derivedbased on a reference block having the smallest SAD in the search area.The derived motion information may be signaled to the decoding apparatusaccording to various methods based on the inter prediction mode.

The coding apparatus performs inter prediction based on the motioninformation on the current block (S620). The coding apparatus may deriveprediction sample(s) for the current block based on the motioninformation. The current block including the prediction samples may bereferred to as a predicted block.

When the merge mode is applied, the motion information of the currentprediction block is not directly transmitted, and the motion informationof the current prediction block is derived using motion information of aneighboring prediction block. Therefore, the motion information of thecurrent prediction block may be indicated by transmitting flaginformation indicating that the merge mode is used and a merge indexindicating which neighboring prediction blocks are used. The merge modemay be called a regular merge mode.

The encoder must search a merge candidate block used to derive motioninformation of the current prediction block to perform the merge mode.For example, up to five merge candidate blocks may be used, but theembodiment(s) of the present document are not limited thereto. A maximumnumber of the merge candidate blocks may be transmitted in a sliceheader or a tile group header and the embodiment(s) of the presentdocument are not limited thereto. After finding the merge candidateblocks, the encoder may generate a merge candidate list and select amerge candidate block having the smallest cost among them as a finalmerge candidate block.

The merge candidate list may use, for example, five merge candidateblocks. For example, four spatial merge candidates and one temporalmerge candidate may be used. Hereinafter, the spatial merge candidate orthe spatial MVP candidate to be described later may be referred to asSMVP, and the temporal merge candidate or the temporal MVP candidate tobe described later may be referred to as TMVP.

FIG. 7 schematically shows a method for constructing a merge candidatelist according to this document.

The coding apparatus (encoder/decoder) inserts spatial merge candidatesderived by searching for spatial neighboring blocks of the current blockinto the merge candidate list (S700). For example, the spatialneighboring blocks may include a bottom left corner neighboring block, aleft neighboring block, a upper right corner neighboring block, an upperneighboring block, and an upper left corner neighboring block of thecurrent block. However, this is an example, and in addition to theabove-described spatial neighboring blocks, additional neighboringblocks such as a right neighboring block, a bottom neighboring block,and a bottom right neighboring block may be further used as the spatialneighboring blocks. The coding apparatus may detect available blocks bysearching the spatial neighboring blocks based on the priority, and mayderive motion information of the detected blocks as the spatial mergecandidates.

The coding apparatus inserts the temporal merge candidate derived bysearching the temporal neighboring block of the current block into themerge candidate list (S710). The temporal neighboring block may belocated on a reference picture that is a picture different from thecurrent picture in which the current block is located. The referencepicture in which the temporal neighboring block is located may be calleda collocated picture or a col picture. The temporal neighboring blockmay be searched in order of the bottom right corner neighboring blockand the bottom right center block of the co-located block for thecurrent block on the col picture. Meanwhile, when motion datacompression is applied, specific motion information may be stored asrepresentative motion information for each predetermined storage unit inthe col picture. In this case, it is not necessary to store the motioninformation for all the blocks in the predetermined storage unit,thereby obtaining a motion data compression effect. In this case, thepredetermined storage unit may be previously determined, for example, in16×16 sample units, 8×8 sample units, or the like, or size informationon the predetermined storage unit may be signaled from the encoder tothe decoder. When the motion data compression is applied, motioninformation of the temporal neighboring block may be replaced withrepresentative motion information of the predetermined storage unit inwhich the temporal neighboring block is located. That is, in this case,from an implementation point of view, a predetermined value isarithmetically shifted to the right based on coordinates (top leftsample position) of the temporal neighboring block, and thereafter, thetemporal merge candidate may be derived based on motion information ofthe prediction block covering an arithmetically left shifted position.For example, in the case of a sample unit having the predeterminedstorage unit is 2n×2n, if the coordinates of the temporal neighboringblock are (xTnb, yTnb), motion information of the prediction blocklocated at the modified position ((xTnb>>n)<<n), (yTnb>>n)<<n)).Specifically, for example, in case where the predetermined storage unitis a 16×16 sample unit, if the coordinates of the temporal neighboringblock are (xTnb, yTnb), motion information of the prediction blocklocated at modified position ((xTnb>>4)<<4), (yTnb>>4)<<4)) may be usedfor the temporal merge candidate. Or, for example, in case where thepredetermined storage unit is an 8×8 sample unit, if the coordinates ofthe temporal neighboring block are (xTnb, yTnb), motion information ofthe prediction block located at the modified position ((xTnb>>3)<<3),(yTnb>>3)<<3)) may be used for the temporal merge candidate.

The coding apparatus may determine whether the number of current mergecandidates is smaller than the maximum number of merge candidates(S720). The maximum number of merge candidates may be predefined orsignaled from the encoder to the decoder. For example, the encoder maygenerate information on the maximum number of merge candidates, encodethe information, and transmit the encoded information to the decoder inthe form of a bitstream. If the maximum number of merge candidates isfilled up, a subsequent candidate addition process may not be performed.

As a result of the checking, if the number of the current mergecandidates is smaller than the maximum number of merge candidates, thecoding apparatus inserts the additional merge candidate into the mergecandidate list (S730).

As a result of the checking, if the number of the current mergecandidates is not smaller than the number of the maximum mergecandidates, the coding apparatus may terminate the construction of themerge candidate list (S740). In this case, the encoder may select anoptimal merge candidate among merge candidates configuring the mergecandidate list based on a rate-distortion (RD) cost, and signalselection information (ex. merge index) indicating the selected mergecandidate to the decoder. The decoder may select the optimal mergecandidate based on the merge candidate list and the selectioninformation.

The motion information of the selected merge candidate may be used asthe motion information of the current block, and the prediction samplesof the current block may be derived based on the motion information ofthe current block. An encoder may derive residual samples of the currentblock based on the prediction samples, and may signal residualinformation on the residual samples to a decoder. The decoder maygenerate reconstructed samples based on the residual samples and theprediction samples derived based on the residual information, andgenerate a reconstructed picture based thereon as described above.

When the skip mode is applied, the motion information of the currentblock may be derived in the same manner as that of the case where themerge mode is applied. However, when the skip mode is applied, theresidual signal for the corresponding block is omitted, and thusprediction samples may be used as reconstructed samples.

When the MVP mode is applied, a motion vector predictor (mvp) candidatelist may be generated using a motion vector of a reconstructed spatialneighboring block and/or a motion vector of a temporal neighboring block(or Col block). That is, the motion vector corresponding to thereconstructed spatial neighboring block and/or the motion vectorcorresponding to the temporal neighboring block may be used as a motionvector predictor candidate. When bi-prediction is applied, an mvpcandidate list for deriving L0 motion information and an mvp candidatelist for deriving L1 motion information may be generated and usedseparately. The above-described prediction information (or informationon the prediction) may include selection information (ex. MVP flag orMVP index) indicating an optimal motion vector predictor candidateselected from the motion vector predictor candidates included in thelist. In this case, the predictor may select a motion vector predictorof the current block from among the motion vector predictor candidatesincluded in the motion vector candidate list using the selectioninformation. The predictor of the encoding apparatus may obtain a motionvector difference (MVD) between the motion vector of the current blockand the motion vector predictor, encode the same, and output it in abitstream form. That is, the MVD may be obtained as a value obtained bysubtracting the motion vector predictor from the motion vector of thecurrent block. In this case, the predictor of the decoding apparatus mayobtain a motion vector difference included in the information on theprediction and derive the motion vector of the current block by addingthe motion vector difference and the motion vector predictor. Thepredictor of the decoding apparatus may obtain or derive a referencepicture index indicating the reference picture from the information onthe prediction.

FIG. 8 is a flowchart illustrating a method of constructing a motionvector predictor candidate list.

Referring to FIG. 8 , an embodiment may first search for a spatialcandidate block for motion vector prediction and insert it into theprediction candidate list (S800). Thereafter, an embodiment maydetermine whether the number of spatial candidate blocks is less thantwo (S810). For example, in an embodiment, when the number of spatialcandidate blocks is less than 2, a temporal candidate block may besearched for and additionally inserted into the prediction candidatelist (S820), and when the temporal candidate block is unavailable, azero motion vector may be used (S830). That is, the zero motion vectormay be additionally inserted into the prediction candidate list.Thereafter, an embodiment may end the construction of the preliminarycandidate list (S840). Alternatively, according to an embodiment, whenthe number of spatial candidate blocks is not less than two, theconstruction of the preliminary candidate list may be terminated. Here,the preliminary candidate list may indicate an MVP candidate list(S840).

Meanwhile, when the MVP mode is applied, the reference picture index maybe explicitly signaled. In this case, the reference picture indexrefidxL0 for the L0 prediction and the reference picture index refidxL1for the L1 prediction may be separately signaled. For example, when MVPmode is applied and BI prediction is applied, both information onrefidxL0 and information on refidxL1 may be signaled.

When the MVP mode is applied, as described above, the information on theMVD derived from the encoding apparatus may be signaled to the decodingapparatus. The information on the MVD may include, for example,information representing x and y components of the MVD absolute valueand the sign. In this case, information indicating whether the MVDabsolute value is greater than 0 and greater than 1, and the MVDremainder may be signaled step by step. For example, the informationindicating whether the MVD absolute value is greater than 1 may besignaled only when the value of the flag information indicating whetherthe MVD absolute value is greater than 0 is 1.

For example, the information on the MVD may be configured as thefollowing syntax, encoded in the encoding apparatus, and signaled to thedecoding apparatus.

TABLE 3 Descriptor mvd_coding( x0, y0, refList, cpIdx ) { abs_mvd_greater0_flag[ 0 ] ae (v)  abs_mvd_greater0_flag[ 1 ] ae (v) if( abs_mvd_greater0_flag[ 0 ] )   abs_mvd_greater1_flag[ 0 ] ae (v) if( abs_mvd_greater0_flag[ 1 ] )   abs_mvd_greater1_flag[ 1 ] ae (v) if( abs_mvd_greater0_flag[ 0 ] ) {   if( abs_mvd_greater1_flag[ 0 ] )   abs_mvd_minus2[ 0 ] ae (v)   mvd_sign_flag[ 0 ] ae (v)  }  if(abs_mvd_greater0_flag[ 1 ] ) {   if( abs_mvd_greater1_flag[ 1 ] )   abs_mvd_minus2[ 1 ] ae (v)   mvd_sign_flag[ 1 ] ae (v)  } }

For example, in Table 3, the abs_mvd_greater0_flag syntax element mayindicate information on whether the difference MVD is greater than 0,and the abs_mvd_greater1_flag syntax element may indicate information onwhether the difference MVD is greater than 1. Also, the abs_mvd_minus2syntax element may indicate information about a value obtained by −2 tothe difference MVD, and the mvd_sign_flag syntax element may indicateinformation about the sign of the difference MVD. In addition, in Table3, [0] of each syntax element may indicate information on L0, and [1]may indicate information on L1.

For example, MVD[compIdx] may be derived based onabs_mvd_greater0_flag[compIdx]*(abs_mvd_minus2[compIdx]+2)*(1-2*mvd_sign_flag[compIdx]).Here, compIdx (or cpldx) represents an index of each component and mayhave a value of 0 or 1. compIdx 0 may indicte x component and compIdx 1may indicate y component. However, this is merely an example and valuesof each component may be expressed by using a coordinate system otherthan the x and y coordinate systems.

Meanwhile, MVD (MVDL0) for L0 prediction and MVD (MVDL1) for L1prediction may be separately signaled, and the information on MVD mayinclude information on MVDL0 and/or information on MVDL1. For example,when the MVP mode is applied to the current block and BI prediction isapplied, both information on the MVDL0 and information on the MVDL1 maybe signaled.

FIG. 9 is a diagram for describing symmetric motion vector differences(SMVD).

When BI prediction is applied, symmetric MVD may be used inconsideration of coding efficiency. In this case, signaling of some ofthe motion information may be omitted. For example, when symmetric MVDis applied to the current block, information on refidxL0, information onrefidxL1, and information on MVDL1 may not be signaled from the encodingapparatus to the decoding apparatus and may be internally derived. Forexample, when MVP mode and BI prediction are applied to the currentblock, flag information (ex. symmetric MVD flag information orsym_mvd_flag syntax element) indicating whether to apply symmetric MVDmay be signaled, and when the value of the flag information is 1, thedecoding apparatus may determine that symmetric MVD is applied to thecurrent block.

When symmetric MVD mode is applied (i.e., the value of symmetric MVDflag information is 1), information on mvp_10_flag, mvp_11_flag, andMVDL0 may be explicitly signaled and, as described above, signaling ofinformation on refidxL0, information on refidxL1, and information onMVDL1 may be omitted and derived internally. For example, refidxL0 maybe derived as an index indicating a previous reference picture closestto the current picture in POC order in reference picture list 0 (whichmay be called list 0 or L0). refidxL1 may be derived as an indexindicating a next reference picture closest to the current picture inthe POC order in reference picture list 1 (which may be called list 1 orL1). Or, for example, refidxL0 and refidxL1 may both be derived as 0.Or, for example, the refidxL0 and refidxL1 may be derived as minimumindexes having the same POC difference in the relationship with thecurrent picture. Specifically, for example, when [POC of currentpicture]-[POC of first reference picture indicated by refidxL0] is afirst POC difference and [POC of the second reference picture indicatedby refidxL1] is a second POC difference, only if the first POCdifference and the second POC difference are the same, a value ofrefidxL0 indicating the first reference picture may be derived as avalue of refidxL0 of the current block and a value of refidxL1indicating the second reference picture may be derived as a value ofrefidxL1 of the current block. In addition, for example, when there area plurality of sets in which the first POC difference and the second POCdifference are the same, refidxL0 and refidxL1 of a set having theminimum difference may be derived as refidxL0 and refidxL1 of thecurrent block.

Referring to FIG. 9 , reference picture list 0, reference picture list1, and MVDL0 and MVDL1 are shown. Here, MVDL1 is symmetric with MVDL0.

MVDL1 may be derived as negative (−) MVDL0. For example, the final(improved or modified) motion information (motion vector; MV) for thecurrent block may be derived based on the following equation.

$\begin{matrix}\left\{ \begin{matrix}{\left( {{mvx}_{0},{mvy}_{0}} \right) = \left( {{{mvpx_{0}} + {mvdx}_{0}},{{mvpy}_{0} + {mvdy}_{0}}} \right)} \\{\left( {{mvx_{1}},{mvy}_{1}} \right) = \left( {{{mvpx_{1}} - {mvdx}_{0}},{{{mvp}y_{1}} - {mvdy}_{0}}} \right)}\end{matrix} \right. & \left\lbrack {{Equation}1} \right\rbrack\end{matrix}$

In Equation 1, mvx₀ and mvy₀ may represent an x component and a ycomponent of L0 motion information or motion vector for L0 prediction,and mvx₁ and mvy₁ may represent an x component and a y component of L1motion information or motion vector for L1 prediction. Also, mvpx₀ andmvpy₀ may represent the x component and y component of the motion vectorpredictor for L0 prediction, and mvpx₁ and mvpy₁ may represent the xcomponent and y component of the motion vector predictor for L1prediction. Also, mvdx0 and mvdy0 may represent an x component and a ycomponent of a motion vector difference for L0 prediction.

Meanwhile, the MMVD mode is a method of applying motion vectordifference (MVD) to the merge mode, and motion information directly usedto generate prediction samples of the current block (i.e., the currentCU) may be implicitly derived. For example, an MMVD flag (i.e.,mmvd_flag) indicating whether to use MMVD for a current block (i.e., acurrent CU) may be signaled, and MMVD may be performed based on thisMMVD flag. When MMVD is applied to the current block (i.e., whenmmvd_flag is 1), additional information on MMVD may be signaled.

Here, the additional information on the MMVD includes a merge candidateflag (i.e., mmvd_cand_flag) indicating whether the first candidate orthe second candidate in the merge candidate list is used together withthe MVD, and a distance index (i.e., mmvd_distance_idx) for indicatingthe motion magnitude and a direction index (i.e., mmvd_direction_idx)for indicating a motion direction.

In the MMVD mode, two candidates located in the first and second entriesamong the candidates in the merge candidate list (i.e., the firstcandidate or the second candidate) may be used, and the two candidates(i.e., the first candidate or the second candidate) may be used. One ofthem may be used as the base MV. For example, a merge candidate flag(i.e., mmvd_cand_flag) may be signaled to indicate any one of twocandidates (i.e., the first candidate or the second candidate) in themerge candidate list.

Furthermore, distance index (i.e., mmvd_distance_idx) specifies motionmagnitude information and indicate the pre-defined offset from thestarting point. The offset may be added to either horizontal componentor vertical component of starting MV. The relation of distance index andpre-defined offset is specified in the following table.

TABLE 4 MmvdDistane[ x0 ][ y0 ] mmvd_distance_idx tile_group_fpel_mmvd_tile_group_fpel_mmvd_ [ x0 ][ y0 ] enabled_flag = = 0 enabled_flag = = 10 1 4 1 2 8 2 4 16 3 8 32 4 16 64 5 32 128 6 64 256 7 128 512

Referring to Table 4 above, the distance of the MVD (eg, MmvdDistance)is determined according to the value of the distance index (eg,mmvd_distance_idx), and the distance of the MVD (eg, MmvdDistance) maybe derived by using integer sample precision or fractional sampleprecision based on the value of tile_group_fpel_mmvd_enabled_flag. Forexample, when tile_group_fpel_mmvd_enabled_flag is equal to 1, itindicates that the distance of the MVD is derived by using integersample precision in the current tile group (or picture header), and whentile_group_fpel_mmvd_enabled_flag is equal to 0, it indicates that thedistance of the MVD is derived by using fractional sample precision inthe tile group (or picture header). In Table 1, information (flag) for atile group may be replaced with information for a picture header, forexample, tile_group_fpel_mmvd_enabled_flag may be replaced withph_fpel_mmvd_enabled_flag (or ph_mmvd_fullpel_only_flag).

In addition, the direction index (eg, mmvd_direction_idx) indicates thedirection of the MVD with respect to the starting point, and mayindicate four directions as shown in Table 5 below. In this case, thedirection of the MVD may indicate the sign of the MVD. The relationshipbetween the direction index and the MVD sign may be expressed as thefollowing table.

TABLE 5 mmvd_distance_idx MmvdSign MmvdSign [ x0 ][ y0 ] [ x0 ][ y0 ][ 0] [ x0 ][ y0 ][ 1 ] 0 +1 0 1 −1 0 2 0 +1 3 0 −1

Referring to Table 5, the sign of the MVD (eg, MmvdSign) is determinedaccording to the value of the direction index (eg, mmvd_direction_idx),and the sign of the MVD (eg, MmvdSign) may be derived for the L0reference picture and the L1 reference picture.

Based on the above-described distance index (eg, mmvd_distance_idx) anddirection index (eg, mmvd_direction_idx), the offset of the MVD may becalculated using the following equations.

MmvdOffset[x0][y0][0]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][0]  [Equation2]

MmvdOffset[x0][y0][1]=(MmvdDistance[x0][y0]<<2)*MmvdSign[x0][y0][1]  [Equation3]

In Equations 2 and 3, the MMVD distance (MmvdDistance[x0][y0]) and MMVDsigns (MmvdSign[x0][y0][0], MmvdSign[x0][y0][1]) may be derived based onTable 4 and/or Table 5. In summary, in the MMVD mode, a merge candidateindicated by a merge candidate flag (eg, mmvd_cand_flag) is selectedfrom among the merge candidates in the merge candidate list derivedbased on the neighboring blocks, and the selected merge candidate isused as a base candidate (eg, MVP). In addition, motion information (ie,motion vector) of the current block may be derived by adding an MVDderived using a distance index (eg, mmvd_distance_idx) and a directionindex (eg, mmvd_direction_idx) based on the base candidate.

A predicted block for the current block may be derived based on motioninformation derived according to the prediction mode. The predictedblock may include prediction samples (prediction sample array) of thecurrent block. When the motion vector of the current block indicates afractional sample unit, an interpolation procedure may be performed,through which prediction samples of the current block may be derivedbased on reference samples in the fractional sample unit within areference picture. When bi-prediction is applied, prediction samplesderived through weighting or weighted averaging (according to phase) ofprediction samples derived based on L0 prediction (that is, predictionusing reference picture and MVL0 in reference picture list L0) andprediction samples derived based on L1 prediction (that is, predictionusing reference picture and MVL1 in reference picture list L1) may beused as prediction samples of the current block. When bi-prediction isapplied, if the reference picture used for L0 prediction and thereference picture used for L1 prediction are located in differenttemporal directions with respect to the current picture (i.e.,bi-prediction and bidirectional prediction), it may be called truebi-prediction.

As described above, reconstructed samples and reconstructed pictures maybe generated based on the derived prediction samples, and thenprocedures such as in-loop filtering may be performed.

As described above, according to this document, when bi-prediction isapplied to the current block, prediction samples can be derived based ona weighted average. Conventionally, the bi-prediction signal (ie, thebi-prediction samples) can be derived through a simple average of the L0prediction signal (L0 prediction samples) and the L1 prediction signal(L1 prediction samples). That is, the bi-prediction samples were derivedas an average of the L0 prediction samples based on the L0 referencepicture and MVL0 and the L1 prediction samples based on the L1 referencepicture and MVL1. However, according to this document, whenbi-prediction is applied, a bi-prediction signal (bi-prediction samples)can be derived through a weighted average of the L0 prediction signaland the L1 prediction signal as follows.

In the above-described MMVD related embodiments, a method that considersa long-term reference picture in the MVD derivation process of MMVD maybe proposed, thereby maintaining and increasing compression efficiencyin various applications. In addition, the method proposed in theembodiments of this document can be equally applied to SMVD, which is asymmetric MVD technology used in inter mode (MVP mode), in addition tothe MMVD technology used in MERGE.

FIG. 10 is a diagram for describing a method of deriving motion vectorsin inter prediction.

In an embodiment of this document, it uses an MV derivation methodconsidering a long-term reference picture in a motion vector scalingprocess of a temporal motion candidate (a temporal motion candidate, atemporal merge candidate, or a temporal mvp candidate). The temporalmotion candidate may correspond to mvCol (mvLXCol). The temporal motioncandidate may be referred to as a TMVP.

The following table describes the definition of a long-term referencepicture.

TABLE 6 The function LongTermRefPic( aPic, aPb, refIdx, LX), with Xbeing 0 or 1, is defined as follows:  If the picture with index refIdxfrom reference picture list LX of the slice containing prediction blockaPb  in the picture aPic was marked as “used for long term reference” atthe time when aPic was the current  picture, LongTermRefPic( aPic, aPb,refIdx, LX) is equal to 1.  Otherwise, LongTermRefPic( aPic, aPb,refIdx, LX) is equal to 0.

Referring to Table 6 above, if LongTermRefPic(aPic, aPb, refldx, LX) isequal to 1 (true), the corresponding reference picture may be marked asused for long-term reference. For example, a reference picture notmarked as used for long-term reference may be a reference picture markedas used for short-term reference. In another example, a referencepicture not marked as used for long-term reference and not marked asunused may be a reference picture marked as used for short-termreference. Hereinafter, a reference picture marked as used for long-termreference may be referred to as a long-term reference picture, and areference picture marked as used for short-term reference may bereferred to as a short-term reference picture.

The following table describes the derivation of TMVP (mvLXCol).

TABLE 7 When availableFlagLXCol is equal to TRUE, mvLXCol andavailableFlagLXCol are derived as follows:  - If LongTermRefPic(currPic, currCb, refIdxLX, LX ) is not equal to LongTermRefPic( ColPic,colCb,   refIdxCol, listCol ), both components of mvLXCol are set equalto 0 and availableFlagLXCol is set equal   to 0.  - Otherwise, thevariable availableFlagLXCol is set equal to 1, refPicList[ listCol ][refIdxCol ] is set to be   tire picture with reference index refIdxColin the reference picture list listCol of the slice containing coding  block colCb in the collocated picture specified by ColPic, and thefollowing applies:    colPocDiff = DiffPicOrderCnt( ColPic, refPicList[listCol ][ refIdxCol ] )         (8-402)    currPocDiff=DiffPicOrderCnt( currPic, RefPicList[ X ][ refIdxLX ] )          (8-403)  - The temporal motion buffer compression process for collocated motionvectors as specified in    clause 8.5.2.15 is invoked with mvCol asinput, and the modified mvCol as output.   - If RefPicList[ X ] [refIdxLX ] is a long-term reference picture, or colPocDiff is equal tocurrPocDiff,    mvLXCol is derived as follows:    mvLXCol = mvCol                              (8-404)   - Otherwise, mvLXCol is derivedas a scaled version of the motion vector mvCol as follows:    tx = (16384 + ( Abs( td ) >> L )) / td                        (8-405)   distScalcFactor = Clip3( −4096, 4095, (tb * tx + 32 ) >> 6)              (8-406)    mvLXCol = Clip3( −131072, 131071,(distScaleFactor * mvCol +    128 − (distScaleFactor * mvCol >= 0 ) ) >>8 ) )                  (8-407)    where td and tb are derived asfollows:    td = Clip3( −128, 127,colPocDiff)                        (8-408)    tb = Clip3( −128, 127,currPocDiff)                        (8-409)

Referring to FIG. 10 and Table 7, when the type of the reference picturepointed to by the current picture (eg, indicating whether a long-termreference picture (LTRP) or a short-term reference picture (STRP)) isnot equal to the type of the collocated reference picture pointed to bythe collocated picture, the temporal motion vector mvLXCol is not used.That is, when all of them are the long-term reference pictures or theshort-term reference pictures, colMV is derived, otherwise, colMV is notderived. In addition, in the case that all of them are the long-termreference pictures and in the case where the POC difference between thecurrent picture and the reference picture of the current picture is thesame as the POC difference between the collocated picture and thereference picture of the collocated picture, the collocated motionvector can be used as it is without scaling. If it is a short-termreference picture and the POC difference is different, the motion vectorof the collocated block is used after scaled.

In an embodiment of this document, MMVD used in the MERGE/SKIP modesignals a base motion vector index (base MV index), a distance index,and a direction index for one coding block as information for derivingMVD information. In the case of unidirectional prediction, MVD isderived from motion information, and in the case of bidirectionalprediction, symmetric MVD information is generated using a mirroring andscaling method.

In the case of bidirectional prediction, MVD information for L0 or L1 isscaled to generate an MVD of L1 or L0. However, when a long-termreference picture is referred, it requires modification in the MVDderivation process.

FIG. 11 illustrates an MVD derivation process of MMVD according to anembodiment of the present document. The method shown in FIG. 11 may befor a block to which bi-directional prediction is applied.

Referring to FIG. 11 , when the distance to the L0 reference picture andthe distance to the L1 reference picture are the same, the MmvdOffsetderived can be used as the MVD as it is, and the POC differences (POCdifference between the L0 reference picture and the current picture andthe POC difference between the L1 reference picture and the currentpicture) are different, MVD can be derived by scaling or simplemirroring (ie, −1*MmvdOffset) according to the POC difference andwhether it is a long-term or short-term reference picture.

In one example, a method of deriving symmetric MVD using MMVD for ablock to which bi-prediction is applied is not suitable for a blockusing a long-term reference picture. It is difficult to expectperformance improvement. Accordingly, in the following drawings andembodiments, an example is introduced, in which MMVD is not applied whenreference picture types of L0 and L1 are different.

FIG. 12 illustrates an MVD derivation process of MMVD according toanother embodiment of the present document. The method shown in FIG. 13may be for a block to which bi-directional prediction is applied.

Referring to FIG. 12 , a method for deriving MVD may be differentaccording to whether a reference picture referenced by a current picture(or a current slice, a current block) is a long-term reference picture(LTRP) or a short-term reference picture (STRP). In one example, whenthe method of the embodiment according to FIG. 12 is applied, a part ofthe standard document according to the present embodiment may bedescribed as shown in the following table.

TABLE 8 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process arc: - a luma location ( xCb, y Cb ) of thetop-left sample of tire current luma coding block relative to the top- left luma sample of the current picture. - reference indices refIdxL0and refIdxL1, - prediction list utilization flags predFlagL0 andpredFlagL1, Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1 Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: - If bothpredFlagL0 and predFlagL1 arc equal to 1, the following applies:  currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ])   currPocDiffL1 = DiffPicOrderCnt( currPic. RefPicList[ 1 ][ refIdxL1] )  - If currPocDiffL0 is equal to currPocDiffL1 and , the followingapplies:    mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][0 ]    mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]  - Otherwise, ifAbst( currPocDiffL0 ) is greater than or equal to Abs( currPocDiffL1 ),the following   applies:    td = Clip3( −128. 127, currPocDiffL0 )    tb= Clip3( −128, 127, currPocDiffL1 )    tx = ( 16384 + ( Abs( td)>> 1) )/ td    distScaleFactor = Clip3( −4096, 4095, (lb * tx + 32) >> 6 )   mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]   - If RefPicList[ 0 ][ refIdxL0 ] is nota long-term reference piciure andRefPicList[ 1 ][ refIdxL1 ]    is not along-term reference picture, the following applies:    mMvdL1[ 0 ] =Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 0 ] +    128 −(distScaleFaclor * mMvdL0[ 0 ] >= 0) ) >> 8)    mMvdL1[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] +    128 −(distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8)   - Otherwise, IfRefPicList[ 0 ][ refIdxL0 ] is a long-term reference picture and   RefPicList[ 1 ][ refIdxL1 ] is a long-term reference picture, thefollowing applies:    mMvdL1[ 0 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffL1 ) ?     mMvdL0[ 0 ] : −mMvdL0[ 0 ]    mMvdL1[ 0 ] = Sign(currPocDiffL0 ) = = Sign( currPocDiffL1 ) ?     mMvdL0[ 0 ] : −mMvdL0[ 0]   - Otherwise, the following applies:    mMvdL0[ 0 ] = 0    mMvdL0[ 1] = 0    mMvdL1[ 0 ] = 0    mMvdL1[ 1 ] = 0  - Otherwise (Abs(currPocDiffL0 ) is less than Abs( currPocDiffL1 )), the followingapplies:    td = Clip3( −128, 127, currPocDiffL1 )    tb = Clip3( −128,127, currPocDiffL0 )    tx = ( 16384 + ( Abs( td ) >> 1)) / td   distScaleFactor = Clip3(−4096, 4095, (tb * tx + 32 ) >> 6 )   mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]   - If RefPicList[ 0 ][ refIdxL0 ] is nota long-term reference picture and RefPicList [ 1 ][ refldxL1 ]    is nota long-term reference picture, the following applies:    mMvdL0[ 0 ] =Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0 ] +    128 −(distScaleFactor * mMvdL1[ 0 ] >= 0) ) >> 8)    mMvdL0[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, , (distScaleFactor * mMvdL1[ 1 ] +    128 −(distScaleFactor * mMvdL 1 [ l ] >= 0)) >> 8 ) )   - Otherwise, IfRefPicList[ 0 ][ refIdxL0 ] is a long-term reference picture and   RefPicList[ 1 ][ refIdxL1 ] is a long-term reference picture, thefollowing applies:    mMvdL0[ 0 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffL1 ) ?     mMvdL1[ 0 ] : −mMvdL1[ 0 ]    mMvdL0[ 1 ] = Sign(currPocDiffL0 ) = = Sign( currPocDiffL1 ) ?     mMvdL1[ 1 ] : −mMvdL1[ 1]   - Otherwise, the following applies:    mMvdL0[ 0 ] = 0    mMvdL0[ 1] = 0    mMvdL1[ 0 ] = 0    mMvdL1[ 1 ] = 0 - Otherwise (predFlagL0 orpredFlagL1 are equal to 1), the following applies for X being 0 and 1:   mMvdLX[ 0 ] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ] :0    mMvdLX[ 1 ] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ]: 0

FIG. 13 illustrates an MVD derivation process of MMVD according toanother embodiment of the present document. The method shown in FIG. 13may be for a block to which bi-directional prediction is applied.

Referring to FIG. 13 , a method for deriving MVD may be differentaccording to whether a reference picture referenced by a current picture(or a current slice, a current block) is a long-term reference picture(LTRP) or a short-term reference picture (STRP). In one example, whenthe method of the embodiment according to FIG. 13 is applied, a part ofthe standard document according to the present embodiment may bedescribed as shown in the following table.

TABLE 9 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: - a luma location (xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, - reference indices refIdxL0and refIdxL1. - prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMydL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 arc derived as follows: - If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ])    currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refIdxL1] )  - If (RefPicList[ 0 ][ refIdxL0 ] is a long-term reference pictureand RefPicList[ 1 ][ refIdxL1 ] is a   short-term reference picture) or(RefPicList 0 ][ refIdxL0 ] is a short-term reference picture and  RefPicList[ 1 ] [ refIdxL1 ] is a long-term reference picture), thefollowing applies:      mMvdL0[ 0 ] = 0      mMvdL0[ 1 ] = 0     mMvdL1[ 0 ] = 0      mMvdL1[ 1 ] = 0  - Otherwise, the followingapplies:   - If currPocDiffL0 is equal to currPocDiffL1, the followingapplies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = MmvdOffset[ xCb ][yCb ][ 0 ]      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]   -Otherwise, if Abs( currPocDiffL0 ) is greater than or equal to AbS(currPocDiffL1 ), the    following applies:      td = Clip3( −128, 127,currPocDiffL0 )      tb = Clip3( −128, 127, currPocDiffL1 )      tx = (16384 + (Abs(td) >> 1) ) / td      distScaleFactor = Clip3( −4096, 4095,(tb * tx + 32 ) >> 6 )      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]     • If RefPicList[ 0][ refIdxL0 ] is not a long-term reference picture, the followingapplies:       mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor *mMvdL0[ 0 ] +       128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )      mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1] +       128 − (distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8)      •Otherwise, the following applies:       mMvdL1[ 0 ] = Sign(currPocDiffL0 ) = = Sign( currPocDiffL1 ) ?        mMvdL0[ 0 ] :−mMvdL0[ 0 ]       mMvdL1[ 0 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffL1 ) ?        mMvdL0[ 0 ] : −mMvdL0[ 0 ]   - Otherwise (Abs(currPocDiffL0 ) is less than Abs( currPocDiffL1 )), the followingapplies:       td = Clip3( −128, 127, currPocDiffL1 )       tb = Clip3(−128, 127, currPocDiffL0 )       tx = ( 16384 + (Abs( td ) >> 1 ) ) / td      distScaleFactor = Clip3( −4096, 4095, (tb * tx + 32 ) >> 6 )      mMvdL1[ 0 ] = MmvdOffset xCb ][ yCb ][ 0 ]       mMvdL1[ 1 ] =MmvdOffset xCb ][ yCb ][ 1 ]      • If RefPicList[ 0 ][ refIdxL0 ] isnot a long-term reference picture, the following applies:       mMvdL0[0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0 ] +       128 −(distScaleFactor * mMvdL1[ 0 ] >= 0) ) >> 8 )       mMvdL0[ 1 ] = Clip3(−2¹⁵. 2¹⁵ − 1, , (distScaleFactor * mMvdL1[ 1 ] +       128 −(distScaleFactor * mMvdL1[ 1 ] >= 0) ) >> 8 ) )      • Otherwise, thefollowing applies:       mMvdL10[ 0 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffL1 ) ?        mMvdL1[ 0 ] : −mMvdL1[ 0 ]       mMvdL0[ 1 ] =Sign(currPocDiffL0 ) = = Sign(currPocDiffL1) ?        mMvdL1[ 1 ] :−mMvdL1[ 1 ] - Otherwise (predFlagL0 or predFlagL1 are equal to 1), thefollowing applies for X being 0 and 1:      mMvdLX[ 0 ] = ( predFlagLX == 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ] : 0      mMvdLX[ 1 ] = (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0

In summary, the MVD derivation process of MMVD, that does not deriveMVDs when the reference picture types in each direction are different,have been described.

In one embodiment according to the present document, MVD may not bederived in all cases of referencing a long-term reference picture. Thatis, when at least one L0 and L1 reference picture is a long-termreference picture, MVD is set to 0, and MVD can be derived only when ashort-term reference picture is included. It is described in detail inthe following drawings and tables.

FIG. 14 illustrates an MVD derivation process of MMVD according to anembodiment of the present document. The method shown in FIG. 14 may befor a block to which bi-directional prediction is applied.

Referring to FIG. 14 , based on the highest priority condition (RefPicL0!=LTRP && RefPicL1 !=STRP), MVD for MMVD may be derived when the currentpicture (or current slice, current block) refers to only short-termreference pictures. In one example, when the method of the embodimentaccording to FIG. 14 is applied, a part of the standard documentaccording to the present embodiment may be described as shown in thefollowing table.

TABLE 10 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: - a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, - reference indices refIdxL0and refIdxL1, - prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1 Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ] )currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refIdxL1 ] ) - If RefPicList[ 0 ][ refIdxL0 ] is not a short-term reference pictureor RefPicList[ 1 ][ refIdxL1 ] is not   a short-term reference picture,the following applies:     mMvdL0[ 0 ] = 0     mMvdL0[ 1 ] = 0    mMvdL1[ 0 ] = 0     mMvdL1[ 1 ] = 0  - Otherwise, the followingapplies:   - If currPocDiffL0 is equal to currPocDiffL1, the followingapplies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]     mMvdL0[ 1 ]= MmvdOffset[ xCb ][ yCb ][ 1 ]     mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]     mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]   - Otherwise, ifAbs( currPocDiffL0 ) is greater than or equal to Abs( currPocDiffL1 ),the    following applies:     td = Clip3( −128, 127, currPocDiffL0 )    tb = Clip3( −128, 127, currPocDiffL1 )     tx = ( 16384 + ( Abs( td) >> 1 ) ) / td     distScaleFactor = Clip3( −4096. 4095, (tb * tx +32) >> 6)     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]     mMvdL0[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]     mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ −1, (distScaleFactor * mMvdL0[ 0 ] +     128 − ( distScaleFactor *mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 1 ] +     128 − ( distScalcFactor * mMvdL0[ 1] >= 0 ) ) >> 8 )   - Otherwise (Abs( currPocDiffL0 ) is less than Abs(currPocDiffL1 )), the following applies:     td = Clip3( −128. 127,currPocDiffL1 )     tb = Clip3( −128, 127, currPocDiffL0 )     tx = (16384 + ( Abs( td ) >> l ) ) / td     distScaleFactor = Clip3( −4096,4095, (tb * tx + 32) >> 6 )     mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][0 ]     mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]     mMvdL0[ 0 ] =Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0 ] +     128 −(distScaleFactor * mMvdL1[ 0 ] >= 0) ) >> 8 )     mMvdL0[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, , (distScaleFactor * mMvdL1[ 1 ] +     128 −(distScaleFactor * mMvdL 1 [ l ] >= 0) ) >> 8 ) ) -Otherwise (predFlagL0 or predFlagL1 are equal to 1), the following applies for Xbeing 0 and 1:  - If RefPicList[ X ] [ refIdxLX ] is not a short-termreference picture, the following applies:     mMvdLX[ 0 ] = 0    mMvdLX[ 1 ] = 0  - Otherwise, the following applies :    mMvdLX[ 0 ]= (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ] : 0    mMvdLX[ 1 ]= (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0

In the embodiment according to the present document, when referencepicture types in each direction are different, MVD is derived when ashort-term reference picture is obtained, and MVD is derived to 0 when along-term reference picture is included. It is described in detail inthe following drawings and tables.

FIG. 15 shows an MVD derivation process of MMVD according to anembodiment of the present document. The method shown in FIG. 15 may befor a block to which bi-directional prediction is applied.

Referring to FIG. 15 , when the reference picture types in eachdirection are different, MmvdOffset is applied when referring to areference picture (short-term reference picture) that is close to thecurrent picture, and MVD has a value of 0 when referring to a referencepicture (long-term reference picture) that is far from the currentpicture. In this case, a picture close to the current picture can beregarded as having a short-term reference picture, but when the closepicture is a long-term reference picture, mmvdOffset may be applied to amotion vector of a list indicating the short-term reference picture.

TABLE 11   mMvdL0 = 0 mMvdL1 = MmvdOffset mMvdL0 = 0 mMvdL1 = (−1) *MmvdOffset mMvdL0 = MmvdOffset mMvdL1 = 0 mMvdL0 = (−1) * MmvdOffsetmMvdL1 = 0

For example, the four paragraphs included in Table 11 may sequentiallyreplace the lowermost blocks (contents) of the flowchart included inFIG. 15 .

In one example, when the method of the embodiment according to FIG. 15is applied, a part of the standard document according to the presentembodiment may be described as the following table.

TABLE 12 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: - a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to thetop-left luma sample of tire current picture, -reference indicesrefIdxL0 and refIdxL1, - prediction list utilization flags predFlagL0and predFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: - If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ])    currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refIdxL1] )  - If currPocDiffL0 is equal to currPocDiffL1 and, the followingapplies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = MmvdOffset[ xCb ][yCb ][ 0 ]      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]  -Otherwise, if Abs( currPocDiffL0 ) is greater than or equal to Abs(currPocDiffL1 ), the following   applies:      td = Clip3( −128, 127,currPocDiffL0 )      tb = Clip3( −128, 127, currPocDiffL1 )      tx = (16384 + ( Abs( td ) >> 1 ) ) / td      distScaleFactor = Clips ( −4096,4095, (tb * tx + 32 ) >> 6)      mMvdL0[ 0 ] = MmvdOffsct[ xCb ][ yCb ][0 ]      mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]   - If RefPicList[0 ] [ refIdxL0 ] is not a long-term reference picture and RefPicList[ 1] [ refIdxL1 ]    is not a long-term reference picture, tire followingapplies:      mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor *mMvdL0[ 0 ] +      128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )     mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1] +      128 − (distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )   -Otherwise, If RefPicList[ 0 ][ refIdxL0 ] is a long-term referencepicture and    RefPicList[ 1 ][ refIdxL1 ] is a long-term referencepicture, the following applies:      mMvdL1[ 0 ] = Sign( currPocDiffL0 )= = Sign( currPocDiffL1 ) ?       mMvdL0[ 0 ] : −mMvdL0[ 0 ]     mMvdL1[0 ] = Sign( currPocDifL0 ) = = Sign( currPocDiffL1 ) ?       mMvdL0[ 0 ]: −mMvdL0[ 0 ]   - Otherwise, the following applies:      mMvdL0[ 0 ] =0      mMvdL0[ 1 ] = 0      mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]     mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]  - Otherwise (Abs(currPocDiffL0 ) is less titan Abs( currPocDiffL1 ) ), the followingapplies:      td = Clip3( −128, 127, currPocDiffL1 )      tb = Clip3(−128, 127, currPocDiffL0 )      tx = ( 16384 + (Abs( td ) >> 1 ) ) / td     distScaleFactor = Clip3(−4096, 4095, (tb * tx + 32 ) >> 6 )     mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL1[ 1 ] =MmydOffset[ xCb ][ yCb ][ 1 ]   - IfRefPicList[ 0 ][ refIdxL0 ] is not along-term reference picture and RefPicList[ 1 ][ refIdxL1 ]    is not along-term reference picture, the following applies:      mMvdL0[ 0 ] =Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0 ] +      128 −(distScaleFactor * mMvdL1 [ 0 ] >= 0) ) >> 8 )      mMvdL0[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, , (distScaleFactor * mMvdL1[ 1 ] +      128 −(distScaleFactor * mMvdL1 [ 1 ] >= 0) ) >> 8 ) )   - Otherwise, IfRefPicList[ 0 ][ refIdxL0 ] is a long-term reference picture and   RefPicList[ 1 ][ refIdxL1 ] is a long-term reference picture, thefollowing applies:      mMvdL10[ 0 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffL1 ) ?      mMvdL1[ 0 ] : −mMvdL1[ 0 ]      mMvdL0[ 1 ] =Sign( currPocDiffL0 ) = = Sign( currPocDiffL1 ) ?      mMvdL1[ 1 ] :−mMvdL1[ 1 ]   - Otherwise, the following applies:      mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb][ 1 ]      mMvdL1[ 0 ] = 0      mMvdL1[ 1 ] = 0 - Otherwise (prcdFlagL0or predFlagL1 are equal to 1 ), the following applies for X being 0 and1:     mMvdLX[ 0 ] = (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ]: 0     mMvdLX[ 1 ] = (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1] : 0

The following table shows a comparison table between the examplesincluded in this document.

TABLE 13 Embodiment A Embodiment B Embodiment C Embodiment D L0 L1 POCL0 Offset L1 Offset L0 Offset L1 Offset L0 Offset L1 Offset L0 Offset L1Offset Short Short Same Offset Offset Offset Offset Offset Offset OffsetOffset Diff (LO >= L1) Offset Scaled Offset Scaled Offset Scaled OffsetScaled Diff (LO < L1) Scaled Offset Scaled Offset Scaled Offset ScaledOffset Long Long Same Offset Offset Offset Offset 0 0 Offset Offset Diff(LO >= L1) Offset (−1) Offset Offset (−1) Offset 0 0 Offset (−1) OffsetDiff (LO < L1) (−1) Offset Offset (−1) Offset Offset 0 0 (−1) OffsetOffset Short Long Same N/A N/A N/A N/A N/A N/A N/A N/A Diff (LO >= L1)N/A N/A N/A N/A N/A N/A N/A N/A Diff (LO < L1) Offset Offset 0 0 0 0Offset 0 Long Short Same N/A N/A N/A N/A N/A N/A N/A N/A Diff (LO >= L1)Offset (−1) Offset 0 0 0 0 0 Offset Diff (LO < L1) N/A N/A N/A N/A N/AN/A N/A N/A

Referring to Table 13, a comparison between methods of applying anoffset in consideration of reference picture types for MVD derivation ofMMVD described in the embodiments according to FIG. 11 to FIG. 15 isshown. In Table 13, Embodiment A may relate to the existing MMVD,Embodiment B may show the embodiment according to FIG. 11 to FIG. 13 ,Embodiment C may show the embodiment according to FIG. 14 , andEmbodiment D may show the embodiment according to FIG. 15 .

That is, in the embodiment according to FIG. 11 , FIG. 12 , and FIG. 13, the method of deriving MVD only when the reference picture types ofboth directions are the same has been described, and in the embodimentaccording to FIG. 14 , the method of deriving MVD only when bothdirections are short-term reference pictures has been described. In thecase of the embodiment according to FIG. 14 , MVD may be set to 0 in thecase of a long-term reference picture for unidirectional prediction. Inaddition, in the embodiment according to FIG. 15 , a method of derivingMVD in only one direction when reference picture types in bothdirections are different has been described. Differences between theembodiments represent various features of the techniques described inthis document, and it can be understood by those of ordinary skill inthe art that the effects to be achieved by the embodiments according tothis document can be implemented based on the features.

In an embodiment according to this document, when the reference picturetype is a long-term reference picture, a separate process is performed.When a long-term reference picture is included, the POC difference(POCDiff)-based scaling or mirroring does not affect performanceimprovement, so the MmvdOffset value is assigned to the MVD in thedirection having the short-term reference picture, and the value 0 isassigned to the MVD in the direction having the long-term referencepicture. In one example, when this embodiment is applied, a part of astandard document conforming to the present embodiment may be describedas shown in the following table.

TABLE 14 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: - a luma location (xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, - reference indices refIdxL0and refIdxL1, - prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1 Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: - If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:  - IfRefPicList[ 0 ][ refIdxL0 ] is a long-term reference picture orRefPicList[ 1 ] [ refIdxL1 ] is a long-   term reference picture, thefollowing applies:   - If RefPicList[ 0 ][ refIdxL0 ] is a short-termreference picture,      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]     mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = 0     mMvdL1[ 1 ] = 0 - Otherwise, if RefPicList[ 1 ][ refIdxL1 ] is ashort-term reference picture,      mMvdL0 0 ] = 0      mMvdL0 1 ] = 0     mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL1[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]   - Otherwise,      mMvdL0[ 0 ] = 0     mMvdL0[ 1 ] = 0      mMvdL1[ 0 ] = 0      mMvdL1[ 1 ] = 0  -Otherwise, the following applies:    currPocDiffL0 = DiffPicOrderCnt(currPic, RefPicList[ 0 ][ refIdxL0 ] )    currPocDiffL1 =DiffPicOrderCnt( currPic, RefPicList[ 1 ][ refIdxL1 ] )   - IfcurrPocDiffL0 is equal to currPocDiffL, the following applies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb][ 0 ]      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]   Otherwise, ifAbs( currPocDiffL0 ) is greater than or equal to Abs( currPocDiffL1 ),the    following applies:      td = Clip3( −128, 127. currPocDiffL0 )     tb = Clip3( −128, 127, currPocDiffL1 )      tx = (16384 + ( Abs( td) >> 1 ) ) / td      distScaleFactor = Clip3(−4096, 4095, (tb * tx + 32) >> 6 )      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]      mMvdL0[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ]      mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵− 1. (distScaleFactor * mMydL0[ 0 ] +      128 − ( distScaleFactor *mMvdL0[ 0 ] >= 0 ) ) >> 8 )      mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 1 ] +      128 − ( distScaleFactor * mMvdL0[1 ] >= 0 ) ) >> 8 )   - Otherwise (Abs( currPocDiffL0 ) is less thanAbs( currPocDiffL1 )), the following applies:      td = Clip3( −128,127, currPocDiffL1 )      tb = Clip3( −128, 127, currPocDiffL0 )      tx= ( 16384 + (Abs( td ) >> 1 ) ) / td      distScaleFactor = Clip3 (−4096, 4095, (tb * tx + 32 ) >> 6 )      mMvdL1[ 0 ] = MmvdOffset[ xCb][ yCb ][ 0 ]      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]     mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0] +      128 − (distScaleFactor * mMvdL1[ 0 ] >= 0) ) >> 8 )     mMvdL0[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1, , (distScaleFactor * mMvdL1[ 1] +      128 − (distScaleFactor * mMvdL1[ 1 ] >= 0) ) >> 8 ) ) -Otherwise (predFlagL0 or predFlagL1 are equal to 1 ), the followingapplies for X being 0 and 1:  - If RefPicList[ X ][ refIdxLX ] is along-term reference picture     mMvdLX[ 0 ] = 0     mMvdLX[ 1 ] = 0  -Otherwise,     mMvdLX[ 0 ] = (predFlagLX = = 1 ) ? MmvdOffset xCb ][ yCb][ 0 ] : 0     mMvdLX[ 1 ] = (predFlagLX = = 1 ) ? MmvdOffsct xCb ][ yCb][ 1 ] : 0

In another example, a portion of Table 14 may be replaced with thefollowing table. Referring to Table 15, Offset may be applied based on areference picture type other than POCDiff.

TABLE 15 - If both predFlagL0 and predFlagL1 are equal to 1, thefollowing applies:  - If RefPicList[ 0 ][ refIdxL0 ] is a long-termreference picture or   RefPicList[ 1 ][ refIdxL1 ] is a long-termreference picture, the   following applies:   - If RefPicList[ 0 ][refIdxL0 ] is a short-term reference picture,    mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][1 ]    mMvdL1[ 0 ] = −MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =−MmvdOffset[ xCb ][ yCb ][ 1 ]   - Otherwise, if RefPicList[ 1 ][refIdxL1 ] is a short-term reference    picture,    mMvdL0[ 0 ] =−MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] = −MmvdOffset[ xCb ][ yCb][ 1 ]    mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ]   - Otherwise,    mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][1 ]    mMvdL1[ 0 ] = −MmvdOffset[ xCb ][ yCb ][ 0 ]    mMvdL1[ 1 ] =−MmvdOffset[ xCb ][ yCb ][ 1 ]

In another example, a portion of Table 14 may be replaced with thefollowing table. Referring to Table 16, it is possible to always setMmvdOffset to L0 and −MmvdOffset to L1 without considering the referencepicture type.

TABLE 16 - If both predFlagL0 and predFlagL1 are equal to 1, thefollowing applies:  - If RefPicList[ 0 ][ refIdxL0 ] is a long-termreference picture or   RefPicList[ 1 ][ refIdxL1 ] is a long-termreference picture, the   following applies:   mMvdL0[ 0 ] = MmvdOffset[xCb ][ yCb ][ 0 ]   mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ]  mMvdL1[ 0 ] = −MmvdOffset[ xCb ][ yCb ][ 0 ]   mMvdL1[ 1 ] =−MmvdOffset[ xCb ][ yCb ][ 1 ]

According to an embodiment of this document, SMVD in the inter mode maybe performed similarly to MMVD used in the above-described MERGE mode.In the case of bidirectional prediction, whether or not symmetric MVD isderived is signaled from the encoding apparatus to the decodingapparatus, and when the related flag (ex. sym_mvd_flag) is true (or thevalue is 1), The second direction MVD (eg, MVDL1) is derived throughmirroring of the first direction MVD (ex. MVDL0). In this case, scalingfor the first direction MVD may not be performed.

The following tables show syntaxes for a coding unit according to anembodiment of this document.

TABLE 17 Descriptor coding_unit( x0, y0, cbWidth, cbHeight treeType ) { if( sps_smvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = = PRED_BI &&  !inter_affine_flag[ x0 ][ y0 ] && RefIdxSymL0 > −1 && RefIdxSymL1 >−1)   sym_mvd_flag[ x0 ][ y0 ] ae (v)  if( inter_pred_idc[ x0 ][ y0 ] =PRED_L1 ) {   if( NumRefIdxActive[ 0 ] > 1 && !sym_mvd_flag[ x0 ][ y0 ])    ref_idx_l0[ x0 ][ y0 ] ae (v)   mvd_coding( x0, y0, 0, 0 )   if(MotionModelIdc[ x0 ][ y0 ] > 0 )    mvd_coding( x0, y0, 0, 1 )   if(MotionModelIdc[ x0 ][ y0 ] > 1 )    mvd_coding( x0, y0, 0, 2 )  mvp_l0_flag[ x0 ][ y0 ] ae (v)  } else {   MvdL0[ x0 ][ y0 ][ 0 ] = 0  MvdL0[ x0 ][ y0 ][ 1 ] = 0  }  if( inter_pred_idc[ x0 ][ y0 ] !=PRED_L0 ) {   if( NumRefIdxActive[ 1 ] > 1 && !sym_mvd_flag[ x0 ][ y0 ])    ref_idx_l1[ x0 ][ y0 ] ae (v)   if( mvd_l1_zero_flag &&inter_pred_idc[ x0 ][ y0 ] = = PRED_BI ) {    MvdL1[ x0 ][ y0 ][ 0 ] = 0   MvdL1[ x0 ][ y0 ][ 1 ] = 0    MvdCpL1[ x0 ][ y0 ][ 0 ][ 0 ] = 0   MvdCpL1[ x0 ][ y0 ][ 0 ][ 1 ] = 0    MvdCpL1[ x0 ][ y0 ][ 1 ][ 0 ] =0    MvdCpL1[ x0 ][ y0 ][ 1 ][ 1 ] = 0    MvdCpL1[ x0 ][ y0 ][ 2 ][ 0 ]= 0    MvdCpL1[ x0 ][ y0 ][ 2 ][ 1 ] = 0   } else {    if( sym_mvd_flag[x0 ][ y0 ] ) {     MvdL1[ x0 ][ y0 ][ 0 ] = −MvdL0[ x0 ][ y0 ][ 0 ]    MvdL1[ x0 ][ y0 ][ 1 ] = −MvdL0[ x0 ][ y0 ][ 1 ]    } else    mvd_coding( x0, y0, 1, 0 )    if( MotionModelIdc[ x0 ][ y0 ] > 0 )    mvd_coding( x0, y0, 1, 1 )    if( MotionModelIdc[ x0 ][ y0 ] > 1 )    mvd_coding( x0, y0, 1, 2 )    mvp_l1_flag[ x0 ][ y0 ] ae (v)   }  }else {   MvdL1[ x0 ][ y0 ][ 0 ] = 0   MvdL1[ x0 ][ y0 ][ 1 ] = 0  } ...}

TABLE 18 if( sps_smvd_enabled_flag && inter_pred_idc[ x0 ][ y0 ] = =PRED_BI &&  !inter_affine_flag[ x0 ][ y0 ] && RefIdxSymL0 > −1 &&RefIdxSymL1 > −1 )  sym_mvd_flag[ x0 ][ y0 ] ae (v)

Referring to Tables 17 and 18, when interpred_idc==PRED_BI and referencepictures of L0 and L1 are available (eg, RefldxSymL0>−1 &&RefldxSymL1>−1), sym_mvdjflag is signaled.

The following table shows a decoding procedure for MMVD referenceindices according to an example.

TABLE 19 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefldxSymL1specifying the list 0 and hst 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: - The variable currPic specifies the current picture. -RefIdxSymL0 is set equal to −1. - For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:  - When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   -DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0,   -DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] )                     <   DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1. - RefIdxSymL1 is set equal to −1. - For eachindex i with i = 0..NumRefIdxActive[ 1 ] − 1, the following applies:  -When all of the following conditions are true, RefIdxSymL1 is set to i:  - DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0,   -DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] )                     >   DiffPicOrdcrCntf currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1. - When RefIdxSymL0 is equal to −1 orRefIdxSymLl is equal to −1, the following applies:  - For each index iwith i = 0..NumRefIdxActive[ 0 ] − 1, the following applies:   - Whenall of the following conditions are true, RefIdxSymL0 is set to i:    -DiffPicOrdcrCnt( currPic, RefPicList[ 0 ][ i ] ) < 0,    -DiffPicOrderCnt( currPic,RefPicList[ 0 ][ i ] )                    >    DiffPicOrderCnt( currPic, RefPicList 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1.  - For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:   - When all of thefollowing conditions are true, RefIdxSymL1 is set to i:    -DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0,    -DiffPicOrdcrCnt( currPic, RefPicList[ 1 ][ i ] )                    <    DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1.

Referring to Table 19, a procedure for deriving availability ofreference pictures of L0 and L1 is described. That is, if there is areference picture in the forward direction among the L0 referencepictures, the index of the reference picture closest to the currentpicture is set as RefldxSymL0, and the corresponding value is set as thereference index of L0. In addition, when if there is a reference picturein the backward direction among the L1 reference pictures, the index ofthe reference picture closest to the current picture is set asRefldxSymL1, and the corresponding value is set as the reference indexof L1.

Table 20 below shows a decoding procedure for MMVD reference indicesaccording to another example.

TABLE 20 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: - The variable currPic specifies the current picture. -RefIdxSymL0 is set equal to −1. - For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies:  - When all of thefollowing conditions are true, RefIdxSymL0 is set to i:   -DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0.   -DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) <    DiffPicOrderCnt(currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to−1. - RefIdxSymL1 is set equal to −1. - For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:  - When all of thefollowing conditions are true, RefIdxSymL1 is set to i:   -DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0,   -DiffPicOrderCnt( currPic, RefPicList 1 ][ i ] ) >    DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to−1. - When RefIdxSymL0 is equal to −1 or RefIdxSymL1 is equal to −1, thefollowing applies:  - For each index i with i = 0..NumRefIdxActive[ 0 ]− 1, the following applies:   - When all of tire following conditionsare true, RefIdxSymL0 is set to i:    - DiffPicOrderCnt( currPic,RefPicList[ 0 ][ i ] ) < 0,    - DiffPicOrderCnt( currPic, RefPicList[ 0][ i ] ) >     DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ]) or RefIdxSymL0 is equal to −1.  - For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies:   - When all of thefollowing conditions are true, RefIdxSymL1 is set to i:    -DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0,    -DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) <     DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to−1. - When RefIdxSymL0 is not equal to −1 and RefIdxSymL1 is not equalto −1, the following applies  - If (RefPicList[ 0 ][ RefIdxSymL0 ] is along-term reference picture and RefPicList[ 1 ]   [ RefIdxSymL1] is ashort-term reference picture) or (RefPicList[ 0 ][ RefIdxSymL0 ] is ashort-   term reference picture and RefPicList[ 1 ][ RefldxSymL1 ] is along-term reference picture),   RefIdxSymL0 and RefIdxSymL1 are set to−1.

Referring to Table 20, as in the embodiment described with FIG. 11 ,FIG. 12 , and FIG. 13 , when the types of L0 or L1 reference picturesare different, that is, if the reference picture types of L0 and L1 aredifferent after reference index derivation for SMVD, SMVD is not used inorder to prevent SMVD in a case that the long term reference picture andthe short term reference picture are used (refer to the lowermostparagraph of Table 20).

In an embodiment of this document, SMVD may be applied in the inter modesimilar to MMVD used in the merge mode. When a long-term referencepicture is used as in the embodiment described with FIG. 14 , thelong-term reference picture may be excluded from the reference indexderivation process for SMVD as shown in the following table in order toprevent SMVD.

TABLE 21 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1, − RefPicList[ 0 ][ i] is a short-term-reference picture. − RefIdxSymL1 is set equal to −1. −For each index i with i = 0..NumRefIdxActive[ 1 ] −1, the followingapplies: − When all of the following conditions are true, RefIdxSymL1 isset to i: − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0, −DiffPicOrdcrCnt( currPic, RefPicList[ 1 ][ i ] ) > DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1,− RefPicList[ 1 ][ i ] is a short-term-reference picture. − WhenRefIdxSymL0 is equal to −1 or RefIdxSymL1 is equal to −1, the followingapplies: − For each index i with i = 0..NumRefIdxActive[ 0 ] −1, thefollowing applies: − When all of the following conditions are true,RefIdxSymL0 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) >DiffPicOrderCnt( currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) orRefIdxSymL0 is equal to −1, − RefPicList[ 0 ][ i ] is ashort-term-reference picture. − For each index i with i =0..NumRefIdxActive[ 1 ] −1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1, − RefPicList[ 1 ][ i] is a short-term-reference picture.

The following table according to another example of this embodimentshows an example of processing not to apply SMVD when a long-termreference picture is used after reference picture index derivation forSMVD.

TABLE 22 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX withX being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] −1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − RefIdxSymL1 is setequal to −1. − For each index i with i = 0..NumRefIdxActive[ 1 ] −1, thefollowing applies: − When all of the following conditions are true,RefIdxSymL1 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) >DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1. − When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies: − For each index iwith i = 0..NumRefIdxActive[ 0 ] −1, the following applies: − When allof the following conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) > DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − For each index iwith i = 0..NumRefIdxActive[ 1 ] −1, the following applies: − When allof the following conditions are true, RefIdxSymLl is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1. − When RefIdxSymL0is not equal to −1 and RefIdxSymL1 is not equal to −1, the followingapplies − If RefPicList[ 0 ][ RefIdxSymL0 ] is a long-term referencepicture or RefPicList[ 1 ][ RefIdx SymL1 ] is a long-term referencepicture, RefIdxSymL0 and RefIdxSymL1 are set to −1.

In one embodiment of this document, in the colMV derivation process ofTMVP, when the reference picture type of the current picture and thereference picture type of the collocated picture are different, themotion vector MV is set to 0, but the derivation method in the case ofMMVD and SMVD are not same as for TMVP and they need to be unified.

Even when the reference picture type of the current picture is along-term reference picture and the reference picture type of thecollocated picture is a long-term reference picture, the motion vectoruses the value of the collocated motion vector as it is, but MV may beset to 0 in MMVD and SMVD. In this case, TMVP also sets MV to 0 withoutadditional induction.

In addition, even if the reference picture types are different, along-term reference picture having a close distance to the currentpicture may exist. Therefore, instead of setting the MV to 0, colMV maybe used as the MV without scaling.

FIG. 16 is a diagram for describing an SMVD according to an embodimentof this document.

For SMVD derivation, a method such as FIG. 16 may be used. That is, anSMVD may be derived based on a short-term reference picture (STRP)and/or a long-term reference picture (LTRP). If a mirrored L0 MVD isused as an L1 MVD, when types of reference pictures are different, aninaccurate MVD may be derived. The reason for this is that a distanceratio (a distance between a reference picture 0 and a current pictureand a distance between a reference picture 1 and a current picture) isincreased and the correlation of motion vectors for directions isreduced.

According to an embodiment of this document, if the availability ofreference pictures are checked and conditions are satisfied,sym_mvd_flag may be parsed. If sym_mvd_flag is true, an MVD of L1(MVDL1) may be derived as a mirrored MVDL0 (MVD of L0).

The following table illustrates a part of a coding unit syntax accordingto the present embodiment.

A derivation procedure of sym_mvd_flag according to the presentembodiment will be described based on Table 23.

TABLE 23 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − RefIdxSymL1 is setequal to −1. − For each index i with i = 0..NumRefIdxActive[ 1 ] − 1,the following applies: − When all of the following conditions are true,RefIdxSymL1 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) >DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1. − When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies: − For each index iwith i = 0..NumRefIdxActive[ 0 ] − 1, the following applies: − When allof the following conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) > DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − For each index iwith i = 0..NumRefIdxActive[ 1 ] − 1, the following applies: − When allof the following conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.

In the present embodiment, reference picture indices (RefldxSynmLX withX=0, 1) for an SMVD may be derived. RefldxSynmL0 may represent (theindex of) the closest reference picture having a POC smaller than thePOC of a current picture. RefldxSymL1 may represent (the index of) theclosest reference picture having a POC greater than the POC of a currentpicture.

The following table describes a method of deriving reference pictureindices for an SMVD according to the present embodiment in a standarddocument format.

TABLE 24 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − RefIdxSymL1 is setequal to −1. − For each index i with i = 0..NumRefIdxActive[ 1 ] − 1,the following applies: − When all of the following conditions are true,RefIdxSymL1 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) >DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1. − When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies: − For each index iwith i = 0..NumRefIdxActive[ 0 ] − 1, the following applies: − When allof the following conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] ) > DiffPicOrderCnt( currPic, RefPicList[0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − For each index iwith i = 0..NumRefIdxActive[ 1 ] − 1, the following applies: − When allof the following conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] ) < DiffPicOrderCnt( currPic, RefPicList[1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.

The following table illustrates comparison results between embodiments.MVD accuracy in an SMVD can be improved by considering a referencepicture type according to the embodiments included in Table 24. In Table25, an MVD may represent MVD 0 (MVD of L0).

TABLE 25 Embodiment P Embodiment Q Embodiment R L0 L1 L0 MVD L1 MVD L0MVD L1 MVD L0 MVD L1 MVD short Short MVD (−1) MVD MVD (−1) MVD MVD (−1)MVD long Long MVD (−1) MVD MVD (−1) MVD short Long MVD (−1) MVD longshort MVD (−1) MVD

Referring to Table 25, the embodiment P illustrates the existing SMVDderivation method. In the embodiment Q, if reference picture types(e.g., STRP/LTRP or LTRP/STRP) mixed in L0 and L1 are used, an SMVD maybe limited. In the embodiment R, if a long-term reference picture (LTRP)is referred, an SMVD may be limited.

The following table describes a method of deriving reference pictureindices for an SMVD according to the embodiment Q of Table 25 in astandard document form.

TABLE 26 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] )                  < DiffPicOrderCnt(currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1.− RefIdxSymL1 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] )                  > DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.− When RefIdxSymL0 is equal to −1 or RefIdxSymL1 is equal to −1, thefollowing applies: − For each index i with i = 0..NumRefIdxActive[ 0 ] −1, the following applies: − When all of the following conditions aretrue, RefIdxSymL0 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 0][ i ] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ])                 > DiffPicOrderCnt( currPic, RefPicList[ 0 ][RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − For each index i with i= 0..NumRefIdxActive[ 1 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] )                 < DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.− When RefIdxSymL0 is not equal to −1 and RefIdxSymL1 is not equal to−1, the following applies − If (RefPicList[ 0 ][ RefIdxSymL0 ] is along-term reference picture and RefPicList[ 1 ][ RefIdxSymL1 ] is ashort term reference picture) or (RefPicList[ 0 ][ RefIdxSymL0 ] is ashort term reference picture and RefPicList[ 1 ][ RefIdxSymL1 ] is along term reference picture), RefIdxSymL0 and RefIdxSymL1 are set equalto 1.

The following tables describe method of deriving reference pictureindices for an SMVD according to the embodiment Q of Table 25 in astandard document form.

TABLE 27 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] > 0, DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] )                  < DiffPicOrderCnt(currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1,− RefPicList[ 0 ][ i ] is a short-term-reference picture. − RefIdxSymL1is set equal to −1. − For each index i with i = 0..NumRefIdxActive[ 1 ]− 1, the following applies: − When all of the following conditions aretrue, RefIdxSymL1 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 1][ i ] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ])                  > DiffPicOrderCnt( currPic, RefPicList[ 1 ][RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1, − RefPicList[ 1 ][ i ] isa short-term-reference picture. − When RefIdxSymL0 is equal to −1 orRefIdxSymL1 is equal to −1, the following applies: − For each index iwith i = 0..NumRefIdxActive[ 0 ] − 1, the following applies: − When allof the following conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) < 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] )                  > DiffPicOrderCnt(currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1,− RefPicList[ 0 ][ i ] is a short-term-reference picture. − For eachindex i with i = 0..NumRefIdxActive[ 1 ] − 1, the following applies: −When all of the following conditions are true, RefIdxSymL1 is set to i:− DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] )                  <DiffPicOrderCnt( currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) orRefIdxSymL1 is equal to −1, − RefPicList[ 1 ][ i ] is a short termreference picture.

TABLE 28 8.3.5 Decoding process for symmetric motion vector differencereference indices Output of this process are RefIdxSymL0 and RefIdxSymL1specifying the list 0 and list 1 reference picture indices for symmetricmotion vector differences, i.e., when sym_mvd_flag is equal to 1 for acoding unit. The variable RefIdxSymLX with X being 0 and 1 is derived asfollows: − The variable currPic specifies the current picture. −RefIdxSymL0 is set equal to −1. − For each index i with i =0..NumRefIdxActive[ 0 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL0 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 0 ][ i ] )                  < DiffPicOrderCnt(currPic, RefPicList[ 0 ][ RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1.− RefIdxSymL1 is set equal to −1. For each index i with i =0..NumRefIdxActive[ 1 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) < 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] )                  > DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.− When RefIdxSymL0 is equal to −1 or RefIdxSymL1 is equal to −1, thefollowing applies: − For each index i with i = 0..NumRefIdxActive[ 0 ] −1, the following applies: − When all of the following conditions aretrue, RefIdxSymL0 is set to i: − DiffPicOrderCnt( currPic, RefPicList[ 0][ i ] ) < 0, − DiffPicOrderCnt( currPic, RefPicList[ 0 ][ i ])                 > DiffPicOrderCnt( currPic, RefPicList[ 0 ][RefIdxSymL0 ] ) or RefIdxSymL0 is equal to −1. − For each index i with i= 0..NumRefIdxActive[ 1 ] − 1, the following applies: − When all of thefollowing conditions are true, RefIdxSymL1 is set to i: −DiffPicOrderCnt( currPic, RefPicList[ 1 ][ i ] ) > 0, − DiffPicOrderCnt(currPic, RefPicList[ 1 ][ i ] )                 < DiffPicOrderCnt(currPic, RefPicList[ 1 ][ RefIdxSymL1 ] ) or RefIdxSymL1 is equal to −1.− When RefIdxSymL0 is not equal to −1 and RefIdxSymL1 is not equal to−1, the following applies: If RefPicList[ 0 ][ RefIdxSymL0 ] is a longterm reference picture or RefPicList[ 1 ][ RefIdxSymL1 ] is a long-termreference picture, RefIdxSymL0 and RefIdxSymL1 are set to −1.

Referring to Table 27 and/or Table 28, if a long-term reference picture(LTRP) is referred, an SMVD may be limited. For example, referring toTable 27, in a reference picture checking procedure, a long-termreference picture may be excluded. Accordingly, other reference pictures(e.g., not a long-term reference picture) may be considered for an SMVD.Referring to Table 28, if a reference picture closest to a currentpicture is a long-term reference picture, an SMVD may not be performed.For example, although short-term reference pictures are included in areference picture list, if a reference picture closest to a currentpicture is a long-term reference picture, an SMVD may not be performed.

In an embodiment of this document, in the MMVD procedure, if the POC ofL0 distance is the POC of L1 or higher, an L1 MVD may be derived as ascaled or mirrored L0 MVD. In the MMVD procedure, if the POC of L0distance is smaller than the POC of L1, in the MMVD procedure, an L0 MVDmay be derived as a scaled or mirrored L1 MVD.

FIG. 17 is a flowchart illustrating an MMVD derivation method accordingto an embodiment of this document.

In an embodiment of this document, an MVD may be derived from an MMVD byconsidering POC differences and/or reference picture types. Referring toFIG. 17 , currPocDiffLX may mean a difference between the POC of acurrent picture and the POC of a reference picture LX. CurrPocDiffL0 andcurrPocDiffL1 may be compared with each other, and types of referencepictures may be checked (“refPicList0 !=LTRP” or “refPicList1 !=LTRP”).By considering conditions, MmvdOffset (derived using mmvd_cand_flag,mmvd_distance_idx, and/or mmvd_direction_idx) may be assigned as thesame value as mMvdLX, a mirrored value or a scaled value.

The following table illustrates a part of a standard document accordingto the present embodiment.

TABLE 29 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: − a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, − reference indices refIdxL0and refIdxL1, − prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: − If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ] )(8-336) currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][refIdxL1 ] ) (8-337) − If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies: mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-338)mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ l ] (8-339) mMvdL1[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340) mMvdL1[ 1 ] = MmvdOffset[ xCb ][yCb ][ l ] (8-341) − Otherwise, if Abs( currPocDiffL0 ) is greater thanor equal to Abs( currPocDiffL1 ), the following applies: td = Clip3(−128, 127, currPocDiffL0 ) (8-342) tb = Clip3( −128, 127, currPocDiffL1) (8-343) tx = ( 16384 +( Abs( td ) >> 1 ) ) / td (8-344)distScaleFactor = Clip3( −4096, 4095, ( tb * tx + 32) >> 6 ) (8-345)mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-346) mMvdL0[ 1 ] =MmvdOffset[ xCb ][ yCb ][ 1 ] (8-347) − If RefPicList[ 0 ][ refIdxL0 ]is not a long-term reference picture and RefPicList[ 1 ][ refIdxL1 ] isnot a long-term reference picture, the following applies: mMvdL1[ 0 ] =Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 0 ] + (8-348) 128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 ) mMvdL1[ 1 ] = Clip3( −2¹⁵,2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] + (8-349) 128 − (distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 ) − Otherwise, the followingapplies: mMvdL1[ 0 ] = Sign( currPocDiffL0 ) = = Sign( currPocDiffL1 ) ?mMvdL0[ 0 ] : −mMvdL0[ 0 ] (8-350) mMvdL1[ 1 ] = Sign( currPocDiffL0 ) == Sign( currPocDiffL1 ) ? mMvdL0[ 1 ] : −mMvdL0[ 1 ] (8-351) − Otherwise(Abs( currPocDiffL0 ) is less than Abs( currPocDiffL1 )), the followingapplies: td = Clip3( −128, 127, currPocDiffL1 ) (8-352) tb = Clip3(−128, 127, currPocDiffL0 ) (8-353) tx = ( 16384 + (Abs( td ) >> 1 ) ) /td (8-354) distScaleFactor = Clip3( −4096, 4095, ( tb * tx + 32 ) >> 6 )(8-355) mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-356) mMvdL1[ l ]= MmvdOffset[ xCb ][ yCb ][ l ] (8-357) − If RefPicList[ 0 ][ refIdxL0 ]is not a long-term reference picture and RefPicList[ 1 ][ refIdxL1 ] isnot a long-term reference picture, the following applies: mMvdL0[ 0 ] −Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0 ] + (8-358) 128 −(distScaleFactor * mMvdL1[ 0 ] >= 0) ) >> 8 ) mMvdL0[ 1 ] = Clip3( −2¹⁵,2¹⁵ − 1, , (distScaleFactor * mMvdL1[ 1 ] + (8-359) 128 −(distScaleFactor * mMvdL1[ l ] >= 0) ) >> 8 ) ) − Otherwise, thefollowing applies: mMvdL10[ 0 ] = Sign( currPocDiffL0 ) = = Sign(currPocDiffL1 ) ? mMvdL1[ 0 ] : −mMvdL1[ 0 ] (8-360) mMvdL0[ 1 ] = Sign(currPocDiffL0 ) = = Sign( currPocDiffL1 ) ? mMvdL1[ l ] : −mMvdL1[ l ](8-361) − Otherwise ( predFlagL0 or predFlagL1 are equal to 1 ), thefollowing applies for X being 0 and 1: mMvdLX[ 0 ] = ( predFlagLX = = 1) ? MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-362) mMvdLX[ 1 ] = ( predFlagLX= = 1 ) ? MmvdOffset[ xCb ][ yCb ][ l ] : 0 (8-363)

If a current picture refers to one or more long-term reference pictures(LTRP), a mirroring procedure in which a POC distance is considered maynot be necessary. The reason for this is that a mirroring MVD obtainedfrom a reference picture at a very long distance compared to another MVDis not effective in an accuracy aspect. Schemes for solving this will bedescribed below.

The following table illustrates comparison results between embodiments.

TABLE 30 Embodiment X Embodiment Y Embodiment Z L0 L1 POC L0 Offset L1Offset L0 Offset L1 Offset L0 Offset L1 Offset Short Short Same OffsetOffset Offset Offset Offset Offset Diff (L0 >= L1) Offset Scaled OffsetScaled Offset Scaled Diff (L0 < L1) Scaled Offset Offset Offset OffsetScaled Long Long Same Offset Offset Offset Offset Offset Offset Diff(L0 >= L1) Offset (−1) Offset Offset (−1) Offset Offset (−1) Offset Diff(L0 < L1) (−1) Offset Offset Offset (−1) Offset Offset (−1) Offset ShortLong Same N/A N/A N/A N/A N/A N/A Diff (L0 >= L1) Offset (−1) OffsetOffset (−1) Offset Offset (−1) Offset Diff (L0 < L1) (−1) Offset OffsetOffset (−1) Offset Offset (−1) Offset Long Short Same N/A N/A N/A N/AN/A N/A Diff (L0 >= L1) Offset (−1) Offset Offset (−1) Offset Offset(−1) Offset Diff (L0 < L1) (−1) Offset Offset Offset (−1) Offset Offset(−1) Offset

Referring to Table 30, the embodiment X illustrates the existing MMVDderivation method. In the embodiment Y, if one or more long-termreference pictures are referred by a current block, an MMVD proceduremay be limited. That is, in the embodiment Y, a comparison procedure ofa POC distance for a long-term reference picture may be omitted. In theembodiment Z, with respect to all the cases, an MMVD derivationprocedure may be limited. That is, in the embodiment Z, with respect toall the cases, a procedure for comparing POC distances may be omitted.In Table 30, the offset may denote MmvdOffset.

FIG. 18 is a flowchart illustrating an MMVD derivation method accordingto an embodiment of this document. The flowchart of FIG. 18 mayrepresent an MMVD derivation method according to the aforementionedembodiment Y.

Referring to FIG. 18 , if a reference picture type is a long-termreference picture, conditions in which POC differences are compared maybe removed, and an anchor MVD used for a mirroring procedure may befixed to L0 MVD.

The following table describes a method of deriving MMVDs according tothe embodiment Y of Table 30 in a standard document form.

TABLE 31 8.5.2.7 Derivation process for merge motion vector differenceinputs to this process are: − a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, − reference indices refIdxL0and refIdxL1, − prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: − If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ]) (8-336)    currPocDiffL1 − DiffPicOrderCnt( currPic, RefPicList[ 1 ][refIdxL1 ] ) (8-337) − If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-338)      mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ l ] (8-339)     mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340)      mMvdL1[ 1] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-341) − If RefPicList[ 0 ][ refIdxL0] is a long-term reference picture or RefPicList[ 1 ][ refIdxL1 ] is along- term reference picture, the following applies      mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-342)      mMvdL0[ 1 ] = MmvdOffset[ xCb][ yCb ][ l ] (8-343)      mMvdL1[ 0 ]= −mMvdL0[ 0 ]      (8-344)     mMvdL1[ 1 ]= −MvdL0[ 1 ]      (8-345) − Otherwise, the followingapplies :      td = Clip3( −128, 127, currPocDiffL0 ) (8-346)      tb =Clip3( −128, 127, currPocDiffL1 ) (8-347)      tx = ( 16384 + ( Abs( td) >> l ) ) / td (8-348)      distScaleFactor = Clip3( −4096, 4095, (tb * tx − 32 ) >> 6 ) (8-349) − if Abs( currPocDiffL0 ) is greater thanor equal to Abs( currPocDiffL1 ), the following applies:      mMvdL0[ 0] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-350)      mMvdL0[ 1 ] = MmvdOffset[xCb ][ yCb ][ 1 ] (8-351)      mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 0 ] + (8-352)                   128 − (distScaleFactor * mMvdL0[ 0 ] >= 0 ) ) >> 8 )      mMvdL1[ 1 ] = Clip3(−2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL0[ 1 ] + (8-353)                  128 − ( distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 )− Otherwise (Abs( currPocDiffL0 ) is less than Abs( currPocDiffL1 )),the following applies:      mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-354)      mMvdL1[ 1 ] = MmvdOffset[ xCb ][ yCb ][ l ] (8-355)     mMvdL0[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor * mMvdL1[ 0] + (8-356)                   128 − (distScaleFactor * mMvdL1[ 0 ] >= 0)) >> 8 )      mMvdL0[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1, , (distScaleFactor *mMvdL1[ 1 ] + (8-357)                   128 − (distScaleFactor * mMvdL1[l ] >= 0) ) >> 8 ) ) − Otherwise ( predFlagL0 or predFlagL1 are equal to1 ), the following applies for X being 0 and 1: mMvdLX[ 0 ] = (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-358) mMvdLX[ 1] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ l ] : 0 (8-359)

FIG. 19 is a flowchart illustrating an MMVD derivation method accordingto an embodiment of this document. The flowchart of FIG. 19 mayrepresent an MMVD derivation method according to the aforementionedembodiment Z.

Referring to FIG. 19 , in the embodiment Z, with respect to all thecases, an MMVD derivation procedure may be limited. With respect to allthe cases, conditions in which POC differences are compared may beremoved, and an anchor MVD used for a mirroring or scaling procedure maybe fixed to L0 MVD.

The following table describes a method of deriving MMVDs according tothe embodiment Z of Table 30 in a standard document form.

TABLE 32 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: − a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, − reference indices refIdxL0and refIdxL1, − prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: − If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:   currPocDifL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ]) (8-336)    currPocDifL1 − DiffPicOrderCnt( currPic, RefPicList[ 1 ][refIdxL1 ] ) (8-337) − if currPocDiffL0 is equal to currPocDiffL1, thefollowing applies:      mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ](8-338)      mMvdL0[ 1 ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-339)     mMvdL1[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340)      mMvdL1[ 1] = MmvdOffset[ xCb ][ yCb ][ l ] (3-341) − If RefPicList[ 0 ][ refIdxL0] is not a long-term reference picture and RefPicList[ 1 ][ refIdxL1 ]is not a long-term reference picture, the following applies:     mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-342)      mMvdL0[ 1] − MmvdOffset[ xCb ][ yCb ][ 1 ] (8-343)      mMvdL1[ 0 ]= −mMvdL0[ 0 ](8-344)      mMvdL1[ 1 ]= −mMvdL0[ 1 ] (8-345) − Otherwise, thefollowing applies:      td = Clip3( −128, 127, currPocDiffL0 ) (8-346)     tb = Clip3( −128, 127, currPocDiffL1 ) (8-347)      tx − ( 16384 +( Abs( td ) >> l ) ) / td (8-348)      distScaleFactor = Clip3( −4096,4095, ( tb * tx + 32 ) >> 6 ) (8-349)      mMvdL0[ 0 ] = MmvdOffset[ xCb][ yCb ][ 0 ] (8-350)      mMvdL0[ 1 ] = MmvdOftset[ xCb ][ yCb ][ 1 ](3-351)      mMvdL1[ 0 ] = Clip3( −2¹⁵, 2¹⁵ − 1, (distScaleFactor *mMvdL0[ 0 ] + (8-352)                   128 − ( distScaleFactor *mMvdL0[ 0 ] >= 0 ) ) >> 8 )      mMvdL1[ 1 ] = Clip3( −2¹⁵, 2¹⁵ − 1,(distScaleFactor * mMvdL0[ 1 ] + (8-353)                   128 − (distScaleFactor * mMvdL0[ 1 ] >= 0 ) ) >> 8 ) − Otherwise ( predFlagL0or predFlagL1 are equal to 1 ), the following applies for X being 0 and1:     mMvdLX[ 0 ] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0] : 0 (8-354)     mMvdLX[ 1 ] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb][ yCb ][ l ] : 0 (8-355)

Furthermore, In an example of the present embodiment, with respect toall the cases, conditions in which POC differences are compared may beremoved, and only a mirroring method may be used. The following tabledescribes a method of deriving MMVDs in this example in a standarddocument form.

TABLE 33 8.5.2.7 Derivation process for merge motion vector differenceInputs to this process are: − a luma location ( xCb, yCb ) of thetop-left sample of the current luma coding block relative to the top-left luma sample of the current picture, − reference indices refIdxL0and refIdxL1, − prediction list utilization flags predFlagL0 andpredFlagL1. Outputs of this process are the luma merge motion vectordifferences in 1/16 fractional-sample accuracy mMvdL0 and mMvdL1. Thevariable currPic specifies the current picture. The luma merge motionvector differences mMvdL0 and mMvdL1 are derived as follows: − If bothpredFlagL0 and predFlagL1 are equal to 1, the following applies:currPocDiffL0 = DiffPicOrderCnt( currPic, RefPicList[ 0 ][ refIdxL0 ] )(8-336) currPocDiffL1 = DiffPicOrderCnt( currPic, RefPicList[ 1 ][refIdxL1 ] ) (8-337) − If currPocDiffL0 is equal to currPocDiffL1, thefollowing applies: mMvdL0[ 0 ] = MmvdOffset[ xCb ][ yCb ][ 0 ] (8-338)mMvdL0[ l ] = MmvdOffset[ xCb ][ yCb ][ 1 ] (8-339) mMvdL1[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-340) mMvdL1[ 1 ] = MmvdOffset[ xCb ][yCb ][ 1 ] (8-341) − Otherwise, the following applies: mMvdL0[ 0 ] =MmvdOffset[ xCb ][ yCb ][ 0 ] (8-346) mMvdL0[ 1 ] = MmvdOffset[ xCb ][yCb ][ 1 ] (8-347) mMvdL1[ 0 ] = −mMvdL0[ 0 ] (8-348) mMvdL1[ 1 ] =−mMvdL0[ 1 ] (8-349) − Otherwise ( predFlagL0 or predFlagL1 are equal to1 ), the following applies for X being 0 and 1: mMvdLX[ 0 ] = (predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 0 ] : 0 (8-358) mMvdLX[ 1] = ( predFlagLX = = 1 ) ? MmvdOffset[ xCb ][ yCb ][ 1 ] : 0 (8-359)

The following drawings are created to explain a specific example of thepresent specification. Since the names of specific devices described inthe drawings or the names of specific signals/messages/fields arepresented by way of example, the technical features of the presentspecification are not limited to the specific names used in thefollowing drawings.

FIGS. 20 and 21 schematically illustrate an example of a video/imageencoding method and related components according to an embodiment(s) ofthis document. The method disclosed in FIG. 20 may be performed by theencoding apparatus disclosed in FIG. 2 . Specifically, for example, inFIG. 20 , S2000 to S2040 may be performed by the predictor 220 of theencoding apparatus, S2050 may be performed by the residual processor 230of the encoding apparatus, and S2060 may be performed by the entropyencoder 240 of the encoding apparatus. The method disclosed in FIG. 20may include the embodiments described in this document.

Referring to FIG. 20 , the encoding apparatus derives an interprediction mode for a current block within a current picture (S2000). Inthis case, the inter prediction mode may include the aforementionedmerge mode, AMVP mode (a mode using motion vector predictor candidates),MMVD, SMVD.

The encoding apparatus derives reference pictures for the interprediction mode (S2010). In an example, the reference pictures may beincluded in the reference picture list 0 (or L0, the reference picturelist L0) or the reference picture list 1 (or L1, the reference picturelist L1). For example, the encoding apparatus may configure referencepicture lists for each of slices included in the current picture.

The encoding apparatus derives motion information for the prediction ofa current block based on the inter prediction mode (S2020). The motioninformation may include reference picture indices and motion vectors.For example, the encoding apparatus may derive reference indices(symmetric motion vector reference indices) for an SMVD. The referenceindices for the SMVD may specify reference pictures for applying theSMVD. The reference indices for the SMVD may include a reference indexL0 (RefldxSumL0) and a reference index L1 (RefldxSumL1).

The encoding apparatus may configure a motion vector predictor candidatelist and may derive a motion vector predictor based on the list. Theencoding apparatus may derive motion vectors based on symmetric MVDs andmotion vector predictors.

The encoding apparatus generates prediction samples based on the motioninformation (S2030). The encoding apparatus may generate predictionsamples based on the motion vectors and the reference picture indicesincluded in the motion information. For example, the prediction samplesmay be generated based on blocks (or samples) specified by a motionvector among blocks (or samples) within reference pictures specified bythe reference picture indices.

The encoding apparatus generates prediction related informationincluding an inter prediction mode (S2040). The prediction relatedinformation may include information about an MMVD, information about anSMVD, etc.

The encoding apparatus derives residual information based on theprediction samples (S2050). Specifically, the encoding apparatus mayderive residual samples based on the prediction samples and the originalsamples. The encoding apparatus may derive residual information based onthe residual samples. In order to derive the residual information, theaforementioned transform and quantization process may be performed.

The encoding apparatus encodes image/video information including theprediction related information and the residual information (S2060). Theencoded image/video information may be outputted in a bitstream form.The bit stream may be transmitted to the decoding apparatus over anetwork or a (digital) storage medium.

The image/video information may include various types of informationaccording to the embodiment of this document. For example, theimage/video information may include the aforementioned informationdisclosed in at least one of Table 1 to Table 33.

In an embodiment, the prediction related information may include interprediction type information specifying whether bi-prediction is appliedto a current block in a current picture. For example, the predictionrelated information may include symmetric motion vector differencereference flag information specifying whether a symmetric motion vectordifference reference is applied based on the inter prediction typeinformation. Furthermore, the reference pictures may include short-termreference pictures. Symmetrical motion vector difference referenceindices may be derived from reference indices indicating the short-termreference pictures based on the symmetric motion vector differencereference flag information. The motion information may include motionvectors for the current block and the symmetric motion vector differencereference indices. The prediction samples are generated based on themotion vectors and the symmetric motion vector difference referenceindices.

In an embodiment, the symmetric motion vector difference referenceindices may be derived based on POC differences between each of theshort-term reference pictures and a current picture. In this case,according to an example, a POC difference between the current pictureand a previous reference picture from the current picture may be greaterthan 0. In another example, a POC difference between a current pictureand a next reference picture from the current picture may be smallerthan 0, but this is merely exemplary.

In an embodiment, the encoding apparatus may configure a referencepicture list L0 (or a reference picture list 0) for L0 prediction and areference picture list L1 (or a reference picture list 0) for L1prediction. For example, the short-term reference pictures may include ashort-term reference picture L0 included in the reference picture listL0 and a short-term reference picture L1 included in the referencepicture list L1. The POC differences may include a first POC differencebetween the short-term reference picture L0 and the current picture anda second POC difference between the short-term reference picture L1 andthe current picture. For example, the symmetric motion vector differencereference indices may include a symmetric motion vector differencereference index L0 and a symmetric motion vector difference referenceindex L1. The symmetric motion vector difference reference index L0 maybe derived based on the first POC difference. The symmetric motionvector difference reference index L1 may be derived based on the secondPOC difference.

In an embodiment, the first POC difference may be the same as the secondPOC difference.

In an embodiment, the encoding apparatus may configure the referencepicture list L0 for L0 prediction. The short-term reference pictures mayinclude a first short-term reference picture L0 and a second short-termreference picture L0 included in the reference picture list L0. Forexample, the POC differences may include a third POC difference betweenthe first short-term reference picture L0 and the current picture and afourth POC difference between the second short-term reference picture L0and the current picture. For example, the symmetric motion vectordifference reference indices may include a symmetric motion vectordifference reference index L0. A reference picture index specifying thefirst short-term reference picture L0 may be used as the symmetricmotion vector difference reference index L0 based on a comparisonbetween the third and fourth POC differences.

In an embodiment, when the third POC difference is smaller than thefourth POC difference, the reference picture index specifying the firstshort-term reference picture L0 may be used as the symmetric motionvector difference reference index L0.

In an embodiment, the image information may include information aboutmotion vector differences (MVD). The motion information may includemotion vectors (MVs). MVDL0 for L0 prediction may be derived based onthe information about the MVD. The MVs may be derived based on the MVDL0and the MVDL1.

In an embodiment, the size of MVDL1 may be the same as the size ofMVDL0. A sign of the MVDL1 may be opposite to that of the MVDL0.

FIGS. 22 and 23 schematically illustrate an example of an image/videodecoding method and related components according to an embodiment(s) ofthis document. The method disclosed in FIG. 22 may be performed by thedecoding apparatus disclosed in FIG. 3 . Specifically, for example, inFIG. 22 , S2200 may be performed by the entropy decoder 310 of thedecoding apparatus, S2210 to S2230 may be performed by a predictor 330of the decoding apparatus, S2240 may be performed by a residualprocessor 320 of the decoding apparatus, and S2250 may be performed byan adder 340 of the decoding apparatus. The method disclosed in FIG. 22may include the embodiments described in this document.

Referring to FIG. 22 , the decoding apparatus receives/obtainsimage/video information (S2200). The decoding apparatus mayreceive/obtain the image/video information through a bit stream. Theimage/video information may include prediction related information(including prediction mode information) and residual information. Theprediction related information may include information about MMVD,information about SMVD, etc. Furthermore, the image/video informationmay include various types of information according to the embodiment ofthis document. For example, the image/video information may include theinformation described along with FIGS. 1 to 19 and/or the informationdisclosed in at least one of Tables 1 to 33.

The decoding apparatus derives an inter prediction mode for a currentblock based on the prediction related information (S2210). In this case,the inter prediction mode may include the aforementioned merge mode,AMVP mode (a mode using motion vector predictor candidates), MMVD, SMVD.

The decoding apparatus derives motion information for the prediction ofthe current block based on the inter prediction mode (S2220). The motioninformation may include reference picture indices and motion vectors.For example, the decoding apparatus may derive reference indices for anSMVD. The reference indices for an SMVD may specify reference picturesfor applying the SMVD. The reference indices for an SMVD may include areference index L0 (RefldxSumL0) and a reference index L1 (RefldxSumL1).

The decoding apparatus may configure a motion vector predictor candidatelist, and may derive a motion vector predictor based on the list. Thedecoding apparatus may derive motion vectors based on symmetric MVDs andmotion vector predictors.

The decoding apparatus generates prediction samples based on the motioninformation (S2230). The decoding apparatus may generate the predictionsamples based on motion vectors included in the motion information andreference picture indices. For example, the prediction samples may begenerated based on blocks (or samples) specified by a motion vectoramong blocks (or samples) within reference pictures specified by thereference picture indices.

The decoding apparatus generates residual samples based on residualinformation (S2240). Specifically, the decoding apparatus may derivequantized transform coefficients based on the residual information. Thequantized transform coefficients may have a one-dimensional vector formbased on a coefficient scan order. The decoding apparatus may derive thetransform coefficients based on a dequantization procedure for thequantized transform coefficients. The decoding apparatus may derive theresidual samples based on an inverse transform procedure for thetransform coefficients.

The decoding apparatus generates reconstructed samples of the currentpicture based on the prediction samples and the residual samples(S2250). The decoding apparatus may further perform a filteringprocedure in order to (modified) reconstructed samples.

In an embodiment, the prediction related information may include interprediction type information specifying whether bi-prediction is appliedto a current block in a current picture. For example, the predictionrelated information may include symmetric motion vector differencereference flag information specifying whether a symmetric motion vectordifference reference is applied based on the inter prediction typeinformation. Furthermore, the reference pictures may include short-termreference pictures. Symmetric motion vector difference reference indicesmay be derived from reference indices indicating the short-termreference pictures based on the symmetric motion vector differencereference flag information. The motion information may include motionvectors for the current block and symmetric motion vector differencereference indices. The prediction samples may be generated based on themotion vectors and the symmetric motion vector difference referenceindices.

In an embodiment, the symmetric motion vector difference referenceindices may be derived based on POC differences between each of theshort-term reference pictures and the current picture. In this case,according to an example, a POC difference between a current picture anda previous reference picture from the current picture may be greaterthan 0. In another example, a POC difference between a current pictureand a next reference picture from the current picture may be smallerthan 0, but this is merely exemplary.

In an embodiment, the encoding apparatus may configure a referencepicture list L0 (or a reference picture list 0) for L0 prediction and areference picture list L1 (or a reference picture list 0) for L1prediction. For example, the short-term reference pictures may include ashort-term reference picture L0 included in the reference picture listL0 and a short-term reference picture L1 included in the referencepicture list L1. The POC differences may include a first POC differencebetween the short-term reference picture L0 and the current picture, anda second POC difference between the short-term reference picture L1 andthe current picture. For example, the symmetric motion vector differencereference indices may include a symmetric motion vector differencereference index L0 and a symmetric motion vector difference referenceindex L1. The symmetric motion vector difference reference index L0 maybe derived based on the first POC difference. The symmetric motionvector difference reference index L1 may be derived based on the secondPOC difference.

In an embodiment, the first POC difference may be the same as the secondPOC difference.

In an embodiment, the encoding apparatus may configure a referencepicture list L0 for L0 prediction. The short-term reference pictures mayinclude a first short-term reference picture L0 and a second short-termreference picture L0 included in the reference picture list L0. Forexample, the POC differences may include a third POC difference betweenthe first short-term reference picture L0 and the current picture and afourth POC difference between the second short-term reference picture L0and the current picture. For example, the symmetric motion vectordifference reference indices may include a symmetric motion vectordifference reference index L0. A reference picture index specifying thefirst short-term reference picture L0 may be used as the symmetricmotion vector difference reference index L0 based on a comparisonbetween the third and fourth POC differences.

In an embodiment, when the third POC difference is smaller than thefourth POC difference, the reference picture index specifying the firstshort-term reference picture L0 may be used as the symmetric motionvector difference reference index L0.

In an embodiment, the image information may include information aboutmotion vector differences (MVD). The motion information may includemotion vectors (MVs). MVDL0 for L0 prediction may be derived based onthe information about MVD. The MVs may be derived based on the MVDL0 andthe MVDL1.

In an embodiment, the size of MVDL1 may be the same as the size ofMVDL0. A sign of the MVDL1 may be opposite to that of the MVDL0.

In the above-described embodiments, the methods are described based onthe flowchart having a series of steps or blocks. The present disclosureis not limited to the order of the above steps or blocks. Some steps orblocks may occur simultaneously or in a different order from other stepsor blocks as described above. Further, those skilled in the art willunderstand that the steps shown in the above flowchart are notexclusive, that further steps may be included, or that one or more stepsin the flowchart may be deleted without affecting the scope of thepresent disclosure.

The methods according to the above-described embodiments of the presentdocument may be implemented in software form, and the encoding deviceand/or decoding device according to the present document is, forexample, may be included in the device that performs the imageprocessing of a TV, a computer, a smart phone, a set-top box, a displaydevice, etc.

When the embodiments in the present document are implemented insoftware, the above-described method may be implemented as a module(process, function, etc.) that performs the above-described function. Amodule may be stored in a memory and executed by a processor. The memorymay be internal or external to the processor, and may be coupled to theprocessor by various well-known means. The processor may include anapplication-specific integrated circuit (ASIC), other chipsets, logiccircuits, and/or data processing devices. Memory may include read-onlymemory (ROM), random access memory (RAM), flash memory, memory cards,storage media, and/or other storage devices. That is, the embodimentsdescribed in the present document may be implemented and performed on aprocessor, a microprocessor, a controller, or a chip. For example, thefunctional units shown in each figure may be implemented and performedon a computer, a processor, a microprocessor, a controller, or a chip.In this case, information on instructions or an algorithm forimplementation may be stored in a digital storage medium.

In addition, the decoding apparatus and the encoding apparatus to whichthe present disclosure is applied may be included in a multimediabroadcasting transmission/reception apparatus, a mobile communicationterminal, a home cinema video apparatus, a digital cinema videoapparatus, a surveillance camera, a video chatting apparatus, areal-time communication apparatus such as video communication, a mobilestreaming apparatus, a storage medium, a camcorder, a VoD serviceproviding apparatus, an Over the top (OTT) video apparatus, an Internetstreaming service providing apparatus, a three-dimensional (3D) videoapparatus, a teleconference video apparatus, a transportation userequipment (i.e., vehicle user equipment, an airplane user equipment, aship user equipment, etc.) and a medical video apparatus and may be usedto process video signals and data signals. For example, the Over the top(OTT) video apparatus may include a game console, a blue-ray player, aninternet access TV, a home theater system, a smart phone, a tablet PC, aDigital Video Recorder (DVR), and the like.

Furthermore, the processing method to which the present document isapplied may be produced in the form of a program that is to be executedby a computer and may be stored in a computer-readable recording medium.Multimedia data having a data structure according to the presentdisclosure may also be stored in computer-readable recording media. Thecomputer-readable recording media include all types of storage devicesin which data readable by a computer system is stored. Thecomputer-readable recording media may include a BD, a Universal SerialBus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, a magnetic tape, afloppy disk, and an optical data storage device, for example.Furthermore, the computer-readable recording media includes mediaimplemented in the form of carrier waves (i.e., transmission through theInternet). In addition, a bitstream generated by the encoding method maybe stored in a computer-readable recording medium or may be transmittedover wired/wireless communication networks.

In addition, the embodiments of the present document may be implementedwith a computer program product according to program codes, and theprogram codes may be performed in a computer by the embodiments of thepresent document. The program codes may be stored on a carrier which isreadable by a computer.

FIG. 24 shows an example of a content streaming system to whichembodiments disclosed in the present document may be applied.

Referring to FIG. 24 , the content streaming system to which theembodiment(s) of the present document is applied may largely include anencoding server, a streaming server, a web server, a media storage, auser device, and a multimedia input device.

The encoding server compresses content input from multimedia inputdevices such as a smartphone, a camera, a camcorder, etc. Into digitaldata to generate a bitstream and transmit the bitstream to the streamingserver. As another example, when the multimedia input devices such assmartphones, cameras, camcorders, etc. directly generate a bitstream,the encoding server may be omitted.

The bitstream may be generated by an encoding method or a bitstreamgenerating method to which the embodiment(s) of the present disclosureis applied, and the streaming server may temporarily store the bitstreamin the process of transmitting or receiving the bitstream.

The streaming server transmits the multimedia data to the user devicebased on a user's request through the web server, and the web serverserves as a medium for informing the user of a service. When the userrequests a desired service from the web server, the web server deliversit to a streaming server, and the streaming server transmits multimediadata to the user. In this case, the content streaming system may includea separate control server. In this case, the control server serves tocontrol a command/response between devices in the content streamingsystem.

The streaming server may receive content from a media storage and/or anencoding server. For example, when the content is received from theencoding server, the content may be received in real time. In this case,in order to provide a smooth streaming service, the streaming server maystore the bitstream for a predetermined time.

Examples of the user device may include a mobile phone, a smartphone, alaptop computer, a digital broadcasting terminal, a personal digitalassistant (PDA), a portable multimedia player (PMP), navigation, a slatePC, tablet PCs, ultrabooks, wearable devices (ex. Smartwatches, smartglasses, head mounted displays), digital TVs, desktops computer, digitalsignage, and the like. Each server in the content streaming system maybe operated as a

1.-17. (canceled)
 18. An image decoding apparatus, comprising: anentropy decoder for receiving image information including predictionrelated information and residual information from a bitstream; apredictor for deriving an inter prediction mode for a current block in acurrent picture based on the prediction related information, forderiving motion information for the current block based on the interprediction mode, and for generating prediction samples based on themotion information; a residual processor for generating residual samplesbased on the residual information; and an adder for generatingreconstructed samples of the current picture based on the predictionsamples and the residual samples, wherein the prediction relatedinformation includes inter prediction type information specifyingwhether bi-prediction is applied to the current block, wherein theprediction related information further includes symmetric motion vectordifference flag information specifying whether symmetric motion vectordifference reference is applied, based on the inter prediction typeinformation, wherein based on the symmetric motion vector differencereference flag information, symmetric motion vector difference referenceindices are derived from reference indices indicating short-termreference picture, wherein the motion information includes motionvectors for the current block and the symmetric motion vector differencereference indices, and wherein the prediction samples are generatedbased on the motion vectors and the symmetric motion vector differencereference indices.
 19. The apparatus of claim 18, wherein the symmetricmotion vector difference reference indices are derived based on pictureorder count (POC) differences between each of the short-term referencepictures and the current picture.
 20. The apparatus of claim 19, furtherincluding configuring a reference picture list L0 for L0 prediction anda reference picture list L1 for L1 prediction, wherein the short-termreference pictures include a short-term reference picture L0 included inthe reference picture list L0 and a short-term reference picture L1included in the reference picture list L1, wherein the POC differencesinclude a first POC difference between the short-term reference pictureL0 and the current picture and a second POC difference between theshort-term reference picture L1 and the current picture, wherein thesymmetric motion vector difference reference indices include a symmetricmotion vector difference reference index L0 and a symmetric motionvector difference reference index L1, wherein the symmetric motionvector difference reference index L0 is derived based on the first POCdifference, and wherein the symmetric motion vector difference referenceindex L1 is derived based on the second POC difference.
 21. Theapparatus of claim 20, wherein the first POC difference is identicalwith the second POC difference.
 22. The apparatus of claim 19, furtherincluding configuring a reference picture list L0 for L0 prediction,wherein the short-term reference pictures include a first short-termreference picture L0 and a second short-term reference picture L0included in the reference picture list L0, wherein the POC differencesinclude a third POC difference between the first short-term referencepicture L0 and the current picture and a fourth POC difference betweenthe second short-term reference picture L0 and the current picture,wherein the symmetric motion vector difference reference indices includea symmetric motion vector difference reference index L0, and wherein areference picture index specifying the first short-term referencepicture L0 is used as the symmetric motion vector difference referenceindex L0 based on a comparison between the third and fourth POCdifferences.
 23. The apparatus of claim 22, wherein when the third POCdifference is smaller than the fourth POC difference, a referencepicture index specifying the first short-term reference picture L0 isused as the symmetric motion vector difference reference index L0. 24.The apparatus of claim 18, wherein the image information includesinformation about motion vector differences (MVD), wherein the motioninformation includes motion vectors (MVs), wherein MVDL0 for L0prediction is derived based on the information about the MVD, whereinMVDL1 for L1 prediction is derived based on the MVDL0, and wherein theMVs are derived based on the MVDL0 and the MVDL1.
 25. The apparatus ofclaim 24, wherein a size of MVDL1 is identical with a size of MVDL0, andwherein a sign of the MVDL1 is opposite to a sign of the MVDL0.
 26. Animage encoding apparatus, comprising: a predictor for deriving an interprediction mode for a current block in a current picture, for derivingreference pictures for the inter prediction mode, for deriving motioninformation for prediction of a current block based on the interprediction mode, for generating prediction samples based on the motioninformation, and for generating prediction related information includingthe inter prediction mode; a residual processor for generating residualinformation based on the prediction samples; and an entropy encoder forencoding image information including the prediction related informationand the residual information, wherein the prediction related informationincludes inter prediction type information specifying whetherbi-prediction is applied to the current block, wherein the predictionrelated information further includes symmetric motion vector differenceflag information specifying whether symmetric motion vector differencereference is applied, based on the inter prediction type information,wherein the reference pictures include short-term reference pictures,wherein based on the symmetric motion vector difference reference flaginformation, symmetric motion vector difference reference indices arederived from reference indices indicating the short-term referencepicture, wherein the motion information includes motion vectors for thecurrent block and the symmetric motion vector difference referenceindices, and wherein the prediction samples are generated based on themotion vectors and the symmetric motion vector difference referenceindices.
 27. The apparatus of claim 26, wherein the symmetric motionvector difference reference indices are derived based on picture ordercount (POC) differences between each of the short-term referencepictures and the current picture.
 28. The apparatus of claim 27, furtherincluding configuring a reference picture list L0 for L0 prediction anda reference picture list L1 for L1 prediction, wherein the short-termreference pictures include a short-term reference picture L0 included inthe reference picture list L0 and a short-term reference picture L1included in the reference picture list L1, wherein the POC differencesinclude a first POC difference between the short-term reference pictureL0 and the current picture and a second POC difference between theshort-term reference picture L1 and the current picture, wherein thesymmetric motion vector difference reference indices include a symmetricmotion vector difference reference index L0 and a symmetric motionvector difference reference index L1, wherein the symmetric motionvector difference reference index L0 is derived based on the first POCdifference, and wherein the symmetric motion vector difference referenceindex L1 is derived based on the second POC difference.
 29. Theapparatus of claim 28, wherein the first POC difference is identicalwith the second POC difference.
 30. The apparatus of claim 27, furtherincluding configuring a reference picture list L0 for L0 prediction,wherein the short-term reference pictures include a first short-termreference picture L0 and a second short-term reference picture L0included in the reference picture list L0, wherein the POC differencesinclude a third POC difference between the first short-term referencepicture L0 and the current picture and a fourth POC difference betweenthe second short-term reference picture L0 and the current picture,wherein the symmetric motion vector difference reference indices includea symmetric motion vector difference reference index L0, and wherein areference picture index specifying the first short-term referencepicture L0 is used as the symmetric motion vector difference referenceindex L0 based on a comparison between the third and fourth POCdifferences.
 31. The apparatus of claim 30, wherein when the third POCdifference is smaller than the fourth POC difference, a referencepicture index specifying the first short-term reference picture L0 isused as the symmetric motion vector difference reference index L0. 32.The apparatus of claim 26, wherein the image information includesinformation about motion vector differences (MVD), wherein the motioninformation includes motion vectors (MVs), wherein MVDL0 for L0prediction is derived based on the information about the MVD, whereinMVDL1 for L1 prediction is derived based on the MVDL0, and wherein theMVs are derived based on the MVDL0 and the MVDL1.
 33. The apparatus ofclaim 32, wherein a size of MVDL1 is identical with a size of MVDL0, andwherein a sign of the MVDL1 is opposite to a sign of the MVDL0.
 34. Atransmitting apparatus of data for an image, comprising: a processorconfigured to obtain a bitstream, wherein the bitstream is generatedbased on deriving an inter prediction mode for a current block in acurrent picture, deriving reference pictures for the inter predictionmode, deriving motion information for prediction of a current blockbased on the inter prediction mode, generating prediction samples basedon the motion information, generating prediction related informationincluding the inter prediction mode, generating residual informationbased on the prediction samples, and encoding image informationincluding the prediction related information and the residualinformation; and a transmitter configured to transmit the datacomprising the bitstream, wherein the prediction related informationincludes inter prediction type information specifying whetherbi-prediction is applied to the current block, wherein the predictionrelated information further includes symmetric motion vector differenceflag information specifying whether symmetric motion vector differencereference is applied, based on the inter prediction type information,wherein the reference pictures include short-term reference pictures,wherein based on the symmetric motion vector difference reference flaginformation, symmetric motion vector difference reference indices arederived from reference indices indicating the short-term referencepicture, wherein the motion information includes motion vectors for thecurrent block and the symmetric motion vector difference referenceindices, and wherein the prediction samples are generated based on themotion vectors and the symmetric motion vector difference referenceindices.